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ABSTRACT 



The study described in this report had tvo main purposes: (l) to 

determine whether the amount and timing of human resources data (iLRD) 
influence <5^:3 ign differentially; (2) to investigate the effect upon 
design of differences in personnel quantity and quality requirements. 
Equipment and HRB inputs (eg, manning quantity and quality, task and 
time line analyses, etc.) which were produced during the development of 
the maintenance equipment of the AGM-69A missile were adapted and pre- 
sented to eight design engineers during four 4-hour sessions. Subjects 
were required to develop a conceptual design of the equipment. The 
experimental design contrasted the simultaneous presentation of all 
HRD inputs and stringent personnel quantity and quality constraints at 
the start of design with the same inputs presented incrementally through- 
out design and "minimal" personnel constraints. Measures of experimental 
differences included: frequency with which engineers selected an auto- 
matic, semi-automatic or manual design solution; number of manual design 
features included in design outputs; number of test sets; and manpower 
required by subjects to exercise their systems. It was found that the 
amount and timing of HKD inputs do exercise some influence on the 
engineer’s design. The personnel requirements imposed did affect design 
decisions. The type of manpower requirement imposed (skill level versus 
personnel number constraints) also appeared to make some difference to 
subjects. Although HRD inputs are responded to by engineers primarily 
when those inputs are phrased as design requirements, infonoational 
inputs (eg, task and time-line analyses) appeared to create an attitude 
of awareness in engineers of personnel requirements. Skill is considered 
by engineers to be of greater significance to system performance than 
numbers of personnel. Engineers display considerable variability in 
their designs; they develop their design concepts quickly and resist 
attempts to modify these concepts. They prefer to receive their HRD 
inputs as early in design as possible. Engineers can and do estimate 
the manning needed to exercise their equipment, but these estimates do 
not always seem to relate to their design concepts, "Hie results of the 
study indicate that, if human resources data are to be incorporated in 
design, these must be supplied at the start of design and they must be 
phrased as design requirements. 



It is recommended that at the very leatt the Request for Proposal 
and the Statement of Work include 

(l) Maximum number of ope rating /maintenance personnel allowed 
by job position; 



Maximum skill level allotted for each job position. 
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( 2 ) 



SUMMARY AND CONCLUSIONS 



PROBLEM 

Human resources data (HRD) inputs supplied during design often fail 
to exercise a significant effect upon that development. It is possible 
that inputs are both insufficient and presented at incorrect times 
during development. The study described in this report had two main 
purposes: (1) to determine whether the amount and timing of HRD influence 
design; and (2) to investigate the effect upon design of diffarent 
personnel quantity and quality requirements. 



APPROACH 

Equipment and HRD inputs which were produced during the development 
of the maintenance equipment of the AGM-69A missile were adapted and 
presented to eight design engineers during four 4-hour sessions. Subjects 
were required to develop a conceptual design of the equipment. The 
experimental study contrasted: (1) the simultaneous presentation of all 
HRD inputs at the start of design with the same inputs presented incre- 
mentally throughout design; and (2) the effect of personnel quantity 
constraint versus the effect of personnel skill constraint. 



RESULTS 

A substantial difference in design outputs was found between the 
experimental conditions. The group receiving all HRD inputs in the 
Statement of Work designed significantly more automatic systems, included 
more taanual features in their designs, and required more and higher 
skilled personnel than the group receiving human resources data inputs 
incrementally. The apparent inconsistency between more automatic systems 
and more manual features is explained as partly a problem of defining the 
nature of automatic systems, since each engineer categorized his own 
design, and partly, that putting all HRD in the Statement of Work made 
the designers more personnel conscious and more concerned with designing 
work into the system for the personnel. The skill-restricted design 
group required fewer and less highly skilled personnel, more semiautomatic 
systems, and more test sets than the quantity-restricted design group. 



CONCLUSIONS 

The amount and timing of human resources data inputs and the type of 
HRD constraints do influence the engineer's design, but not always in a 
predictable manner. The fact that a personnel requirement is imposed on 
engineers does not necessarily mean that they will design to the letter 
of the constraint. However, it does mean that they will take the personnel 
factor into greater account than they would if no personnel requirement 
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had been imposed at all. From this standpoint, the more human resources 
data provided to the designer, the more conscious he will be of the need 
to consider personnel limitations in his design, hence the more his design 
will be affected. 



OPERATIONAL IMPLICATIONS 

It is recommended that Re-quests for Proposal and Statements of Work 
for new systems include 

(1) Maximum number of operating/maintenanc.e personnel to be 
allowed in the crew by job position; 

(2) Maximum skill level allowed for each job position; 

(3) The task capabilities of these personnel; 

(4) The design implications of these requirements in terms of 
system characteristics. 



RESEARCH IMPLICATIONS 

Additional research is needed regarding a number of problems. 

(1) Methods are needed for determining early during the conceptual phase 
of a system the "likely" manpower force to be available during the time 
period when the system would become operational. This "likely" force 
wou‘ .-rived from sources such as phased out systems, career personnel, 

rew „ h .jees, and current training courses. (2) Techniques are needed for 
comparing the "likely" force with alternate forces of varying quantities and 
skill distributions and determining their impact on the cost, capability, 
reliability, availability, etc. of the system, Ultimately a "desired" 
manpower structure would be proposed for the system. (3) The design 
implications of manpower requirements need to be fully developed, so that 
the design concepts and characteristics which will yield the "desired" 
manpower force can be specified in the contract statement of werk. For this 
research to have maximum validity, it should study the performance of 
personnel at operational sites in relation to the design concepts of the 
system. (4) Finally, methods are needed for periodically testing during 
design and development for compliance of design with the manpower 
requirement constraints. 
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SECTION 1 



INTRODUCTION 



A* NATURE OF THE PROBLEM 

Although it is a commonplace that human error contributes signi- 
ficantly to system breakdown or mission failure, it is equally well 
known that human resources data (HRD) supplied during system development 
(ie, those describing the quantity, type and functions of manpower 
needed to operate and maintain an equipment effectively) often fail to 
exercise a significant effect on that development* 

The fact that engineers characteristically do not take human factors 
inputs into consideration during equipment design has been documented 
in a series of studies (Meister and Farr, 1966, Meister and Sullivan, 

1967 > Meister, Sullivan and Askren, 1968, and Snyder and Askren, 1968) . 

It is also reported anecdotally by many human factors specialists 
working on development projects. 

Why does this condition exist? There are several possible expla- 
nations. If the engineer does r.ot make use of certain inputs, it may 
be that these inputs do not contain the information he needs to make 
design decisions; or the inputs may be late in reaching him and her.ce 
cannot influence an already completed design; or else these inputs may 
be formulated in terms which the engineer cannot understand and utilize. 
Underlying these hypotheses is the assumption that, given the engineer's 
eagerness to scrutinize as much information bearing on design as possible 
(a fact which was demonstrated in Meister et al., 1968 ), an input which 
fails to influence design fails to do so because it lacks some charac- 
teristic required by the engineer. 

To study this problem it is necessary to investigate the conditions 
under which HRD can and will be utilized by the engineer. The following 
factors must be examined: 

(1) 3he manner in which the engineer designs, because human 
resources inputs must fit into that process; 

( 2 ) The format or manner in which these inputs are supplied to 
design engineers; 

( 3 ) The timing or sequence with which inputs arc? supplied; 

(4) The desi gn- relevancy of the data suppli- 
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(5) The effect of manpower requirements as constraints on hard- 
ware design; 



( 6 ) The availability of information as a whole to the engineer 
during the design process; 

( 7 ) The engineer’s attitude toward the personnel aspects of the 
system ard to human resources data as inputs to design. 

To study these factors human resources inputs must be presented 
to engineers in different formats under controlled conditions in a 
realistic design context. The designs they produce as a consequence 
of different experimental treatments can then be related to uhe 
factors described in these treatments. 

Before considering the puipose of the present study it s necessary 
to describe the authors’ concept of Human Resources Tata (HRb) • This 
has been expressed graphically in Figure 1 . There are two types of HRD: 
(l) Manpower Requirements (MR) , which specify the maximum number and 
skill levels of perscnnel for whom the system is to be desigred; ( 2 ) 
Support Data (SJ)}, eg. Quantitative and Qualitative Persorne] Require- 
ments Information (QQFRl) , personnel availability, task and iirae line 
analyses and training analyses. MR have -or should have- a direct 
influence on the engineer by requiring him to modify Ms design to meet 
the manpower requirements. 3 D are the backup analyse:: which lead to 
the development of MR, and which also serve as descriptive data explain- 
ing the implications of MP. to the engineer. 

Note that SD do not directly influence design; that function is 
reserved to MR. SD may, however, give rise to Support Requirements when 
its analyses are transformed into descriptions of tasks for T *hich the 
system must be designed and the training which system personnel mubt 
receive • 

It should be noted that the effect of both MR and cuppoit require- 
ments on design must be mediated by the determination of the human 
resources -hardware relationships (design implication s) of these requi La- 
ments. When such implications are not explicitly develcptd end provlh i 
as guidelines to the engineer, the effect of MR and support requirements 
(particularly the latter) is largely nullified. Ip another study 
(Meister ei al., 1969) it has been pointed out that the data describing 
these design implications is largely lacking, which accounts for the 
relatively weak influence quality manpouer requirements and personnel 
support data haw 2 har. on previous design. 
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B. PURPOSE OF THE STORY 



This study had tvo major purposes: ( l) To test the hypothesis 

that the quantity and timing of HRD provided to design 
engineers will influence the nature of their designs; and (2) to 
investigate the effect upon design of differences in MR (manpower 
quantity and quality requirements) . 

He present study is in large part a replication of a previous one 
(Meister et al., 1968) to verify trends uncovered in the earlier study, 
That study sought to determine the effect on system design of using 
MR and SD as design inputs. 

In that study equipment data and personnel inputs, eg, quantity 
and skill level of manning, and task information, vere presented incre- 
mentally to six design engineers in a simulation o^ the Phase lA/lB 
development of the Titan III propellant transfer and pressurization 
subsystem* Subjects were required to create a complete subsystem 
design, including schematics, equipment descriptions, drawings and bills 
of material* 

It was found that MR and SD do influence the equipment config- 
uration, but only moderately, because equipment design proceeds so 
rapidly that HRD inputs presented increme ntally inevitably lag design. 
Engineers were found to be responsive only to inputs which are framed 
as design requirements. Although MR inputs in terras of quantity (ie, 
number of personnel) vere readily grasped by engineers, they experienced 
great difficulty in understanding and utilizing quality, ie;, skill level, 
inputs. The results of the study indicated that if personnel factors 
are to be incorporated into design, HRD inputs must be supplied as 
design requirements to the engineer in the statement of work (SOW) pre- 
ceding design. On the basis of that study the following hypotheses 
seemed in order. 

(l) Design engineers approach their problems from the? staid; with 
preconceived concepts and very rapidly organize their sub- 
system designs in equipment terms. They proceed very quickly 
through such initial system analytic stages as determination 
of subsystem functions, allocation of functions between 
equipment and personnel, and determination of equipment types 
and functional characteristics. This approach appears to be 
characteristic of all types of engineers, even those who are 
highly sophisticated in system analysis. The stages so 
compressed are those to which HRD should contribute, if HRD 
is to have a significant impact on the basic nature of sub- 
system design. 
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(2) Because of the manner in which he designs, the primary source 
of the engineers design decisions appears to be the SOW with 
which he begins design. His design is primarily affected by 
the requirements and constraints expressed in the SOW. 

(3) HxD inputs are utilized if they can be interpreted by the 
engineer in terms of requirements or constraints, or if they 
provide information about the subsystem which implies require- 
ments or constraints. This means that, as was pointed out 
earlier, the design implications of MR and SD must be made 
explicit by the human resources specialist in such a way that 
the engineer readily recognizes them. 

The implications of these hypotheses suggest that HRD inputs will 
be maximally effective if they can be presented to the engineer as 
design requirements or constraints within the SCW, This is in line 
with the concepts expressed by Eckstrand et al, (1968) in their paper 
on the changing philosophy of human resources engineering. Because the 
design engineer so rapidly translates system requirements into hardware 
equivalents, HRD inputs will be effective only to the extent that they 
exercise the control Eckstrand et. al. recognize as the next stage in 
human resources engineering. Where HRD inputs are ineffective, it is 
probably because these inputs fail to exercise as much control as do 
equipment inputs, 

Ihere appears to be some evidence also that design engineers, in 
developing their basic subsystem concepts, have some general ideas of 
the crew which they believe will operate and maintain the subsystem. It 
would be extremely helpful, in defining HRD parameters more precisely 
in the SOW, to investigate in greater detail the nature of the crew 
concept which the engineer utilizes as the basis for his design. This 
would enable investigators to redefine MR in terms which the designer 
will more readily recognize as being design-relevant • 

If the conclusions of that study are valid, then presentation of 
more comprehensive HRD inputs to the engineer in the SOW, including 
rigorous MR requirements , should lead to major differences in resultant 
designs . 

The goals of the study described in this report can therefore be 
phrased as a series of questions to be answered: 

( 1) Will differences in amount and timing of HRD inputs result in 
different design concepts? 

(2) Will quantity (ie, numoer of personnel) constraints produce 
different effects on design than quality (ie, skill level) 
constraints? 
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( 3 ) Will the imposition of MR constraints affect equipment charac- 
teristics after the basic design concept has been developed? 

(4) Will the removal of MR constraints affect equipment character- 
istics after the basic design concept has been developed? 

(5) Which HKD inputs are preferred and utilized by engineers and 
at vhat stage in system development? 

(6) How does the engineer's concept of manpower for his system 
relate to his design and how does this concept relate to more 
objective methods of predicting manpower (eg, Barton et al* T s, 
1964 queuing model) ? 

The ultimate purpose of the present study is to derive from the 
controlled testing of engineers certain human re sources -hardware relation- 
ships which would enable the Air Force to write more effective procurement 
requirements* It is assumed tliat if, instead of general, non -enforceable 
manpower provisions, explicit design -relevant statements of human 
resource needs can be incorporated in procurement requirements, more 
satisfactory equipment will be developed* 
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SECTION II 



TEST METHODOLOGY 



A. GENERAL STRATEGY 

The methodology employed in this study has been described in detail 
in Meister et al.* 1968. Hence only that information needed by the 
reader who is unfamiliar with the preceding study will be presented in 
this section* 

The overall reseaich strategy involves placing the engineer in a 
realistic design situation in which he must solve a series of design 
problems by using equipment and HRD information related to these 
problems. In adapting this general methodology to the present study* 
the following steps were performed: 

(1) Selection of an already existent subsystem which could serve 
as a model for the development of test inputs and outputs, 

( 2 ) Selection of appropriate engineer-subjects skilled in design 
of the type of subsystem selected, 

(3) Determination of the equipment and HRD inputs which are 
characteristically provided during the system definition 
phase of development, 

(4) Development of HRD inputs. 

(5) Determination of the sequence in which HRD inputs should be 
provided. 

(6) Determination of the design responses and outputs which the 
engineer-subjects should apply in attempting to solve the 
design problems. 

( 7 ) Determination of specific measures which could be used to 
answer the questions which initiated the study. 
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DEVELOIMENT OF TEE EXPERIMENTAL SITUATION 



1 • S election of the Test S u bsystem 
A. Rationale 



The initial step in the development of the experimental 
situation vas the selection of a subsystem which already 
existed, if not in hardware form, then in the form of a 
completed design. This subsystem could then be used as a 
irodel to develop the necessary study inputs and outputs. 

The idea of using an already existent subsystem as a 
model for test inputs has been found to be useful, for 
several reasons: 

(1) Both equipment and HRD inputs, the details of which 
would otherwise be difficult to create if one had to 
create them out of imagination, could be abstracted 
from the original documentation. 

(2) The amount of informational detail that should be 
provided at the various stages of the experimental 
subsystem development could be determined from the 
original documentation. 

( 3 ) The face validity (i.e., realism) of the inputs could 
be assured because they were produced in the original 
subsystem design. 

(k) The design responses required of subjects could be 
determined on the basis of the design outputs de- 
veloped in the original subsystem. 

3 • Criteria for Selecting the Model Subsyste m 

The criteria for selection of the model subsystem were 
as follows: 

(l) The subsystem should be one in which personnel 
functioning is important. For this reason it was 
decided to select a maintenance subsystem. S^nce 
many operator subsystems in present Air Force systems 
are highly automated, it was considered that a 
maintenance subsystem would offer a greater amount 

of direct personnel- equipment interaction. 



(2) The subsystem should have an appropriate degree of 
complexity. Overly simple subsystems were avoided 
since the number of HKD inputs and their effect on 
subsystem design would be minimal. At the same time 
an overly complex subsystem would make it difficult 
to supply the necessary design inputs within the time 
schedule established. 

(3) The subsystem should be one whose development pro- 
ceeded in accordance with AFSCM 375-5 (USAF, 1964 ) • 
AFSCM 375-5 vas utilized as a framework for the 
development of the experimental HRD inputs because 
Air Force systems are presently required to be 
developed in the spirit, if not to the letter, of 
AFSCM 375-5. 

(4) The unclassified records of the model subsystem should 
be complete enough to minimize the development of 

new material (as opposed to the editing or revision 
of old material). 

(5) Ihe subsystem should be recently included in the Air 
Force inventory, or under development, so that the 
inputs would take advantage of recent technical 
developments in the state of the art. 

C. The Subsystem Selected 

With these criteria in mind, several alternative subsystems 
were considered and evaluated before the investigators selected 
the model subsystem. 

The subsystem selected for simulation was the unclassified 
aspects of the Aerospace Ground Equipment (AGK) of the AGM- 69 A, 
the SHAM. The AGM- 69 A is an air to ground missile designed to 
be launched from the B-^2 bomber. It is presently under 
development by the Boeing Company, Seattle, Washington. 

The specific equipment to be designed by the engineer 
subjects was the electronic test equipment used to check out 
the status of the missile prior to its installation on the 
aircraft, to troubleshoot the missile if any malfunctions 
were found in pre-installation checkout, and to check out 
Issile- related aircraft systems. Unclassified details of 
the required characteristics of the AGM- 69 A test equipment 
may be found in Appendix I. 



2. Selection of Subjects 



The eight engineers who made up the subject population for 
this study were selected from the test engineering department of 
the Marquardt Corporation, Van Nuys, California. Engineers were 
selected from this company because the design of the maintenance 
equipment for the AGM- 69 A required the selection of personnel 
skilled in the design of test equipment used to check out missiles 
and missile-related equipment. 

An analysis of the education and experience background of the 
subjects is presented in Table I. The subjects are considered to 
be essentially equivalent in terms of relevant experience to those 
of the earlier study (Meister et al., 1968) . They are also con- 
sidered on the basis of their verbal responses during the experiment 
to be similar to the 36 engineers tested earlier in studies described 
in Meister and Farr, 1966, Meister Sullivan, 1967 and Meister 
et al., 1968. The average amount ol experience is 15*7 years, with 
no subject having less than 8. Beyond a certain experience level, 
represented by the present subject group, differences in years of 
experience are felt to have little or no significance for design 
output . 



3 • Determination of Equipment Inputs 
A. Description of Inputs 

In addition to KRD inputs, equipment inputs were provided 
to serve as the context for the KRD inputs as well as the 
information base for the design. These included the following: 

( 1 ) Statement of work which initiated subsystem develop- 
ment • 

(2) System and equipment functional flow diagrams (at 
successive levels of detail) • 

(3) Requirements Allocation Sheets (RAS). 

(4) Descriptions of equipment characteristics • 

Fev changes were made to the original specification for the 
AGM- 69 A test equipment and then only to facilitate its use by 
subjects. Significant changes were made in the phraseology of 
section 5 '7 (Personnel Manning Requirements) to implement the 
various experimental treatments in the study. Changes were also 
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TABLE I 



SUBJECT EDUCATION AND EXPERIENCE 



Subject 


Education 


Years of Experience 


Y 


BSEE 


12 


L 


ESSE 


11 


F 


BS 


IT 


Me 


MS EE 


12 


M 0 


BSEE 


8 


M a 


BSEE 


21 


K 


BSEE 


22 


W 


BS 


23 






15.7 




II 

20 



made to the reliability and maintainability sections of the 
section, since the original specification for the test equip- 
ment did not contain subsystem reliability/maintainability 
requirements. The values supplied were based on those found 
for similar test equipment in other systems. 

To develop th* equipment inputs, unclassified documentation 
produced during the development of the AGM- 69 A was examined, 
courtesy of the Boeing Company and AGM-69A System Project 
Office; pertinent material was extracted and. prepared as shown 
in Appendix I. To ensure technical accuracy and completeness 
of the equipment inputs provided to the subjects, they were 
reviewed by the Chief Design Engineer of The Marquardt 
Corporation, and require! revisions were incorporated. 

All inputs were provided in complete form except where it 
was det, "ed that the subject solve a problem which required 
him to develop or complete some part of the input* For example 
if system functions on requirements Allocation Sheets were to 
be analyzed by the subject to determine appropriate equipment 
characteristics, all necessary data were included on the sheets 
except for those dealing with the equipment characteristics. 
Complete inputs were provided because the designers were not 
expected to be able to develop all the documentation which 
would ordinarily be developed due to the time -scale involved 
in the simulation. Moreover, all HRD inputs were presented in 
teto, since designers do not ordinarily develop such inputs and 
do not have the experience needed to do so. 



B. Input Presentation Ground Rules 

The following ground rules were followed: 

(1) All inputs to subjects were supplied in written form, 
except where immediate c ire unistances (e.g., answers 
to questions asked by the subject during the test 
session) made this impossible. Any input provided 
orally was documented immediately following its 
transmission. 

(2) Instructions to subjects were provided verbally, but 
they were allowed to read the same instructions in 
written form; and those written instructions were 
available to Mm throughout the test session. 
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4. Development of HKD Inputs 



The HRD inputs selected for presentation to subjects were 
those which are developed as a result of analyses performed during 
Phases lA/lB of the System Definition Stage and prior to the 
Acquisition Phase of System Development. Consequently, whether 
provided either as part of the SOW' or incrementally, they were of 
a general nature describing system functions rather than those 
describing molecular human engineering details relevant to hardware 
components. Previous studies have indicate! that the basic design 
is "frozen 11 prior to Acquisition; human engineering inputs therefore 
represent only minor refinements to the Phase LA/lB design. 

According to AFSCM 375 “5 human factors inputs should be avail- 
able prior to the issuance of an RFP to the . contractor. From that 
standpoint it was considered appropriate that for one of the two 
subject group s these inputs should be included in their SOW before 
these subjects began their design. 

In practice, however, the Air Force often delegates to a con- 
tractor the responsibility for developing these inputs after design 
has begun. When this occurs, HBD inputs are usually provided to 
the design engineer on an incremental basis. From that standpoint 
it was considered legitimate to present these inputs to a second 
group of subjects on an incremental basis. 

The HKD inputs provided are listed in Table II and are also 
presented in Appendix I. 

Material supplied in the SOW is listed in Table III. 



5 • Determination of th e S e quence of Providing HRD Inputs 

A. Simulation of the Development Process 

System development, either as formally defined by AK>CM 
375** 5 or as actually practiced, is a process of multiple 
iterations; however, it has been documented (Meister et al., 
1968) that the basic design concept is developed very early 
in the iterative cycle, and that subsequent iterations only 
serve to refine the basic design concept. 

For this reason is was felt that ihe design simulation 
could be compressed into four 4-hour sessions without any 
great loss of precision in the experimental results. For one 
group of subjects (called the Omnibus gioup) all MP and PSD 
inputs were supplied as part of the SW with which they began 
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TABLE II 



LIST AND DEFINITION OF 
HUMAN RESOURCES DATA INPUTS 

I . MANPOWER REQUIR E MENTS 



Item 



Definition 



(1) 


Nximber of personnel 


Quantity of personnel required to 
perform subsystem operations, 
defined in terms of i maximum number 
allowed . 


( 2 ) 


Skill level 


Air Force skill levels allowed for 
the task. 


SUPPORT DATA 






Item 


Definition 


(1) 


Lists of ]>ersonnel tasks 


Tasks defined 5.n terms of personnel 
functions and equipment acted upon. 


( 2 ) 


Personnel/ equipment flow 
diagrams 


Diagrams illustrating the sequencing 
and interrelationships among tasks. 


( 3 ) 


Personnel/equipment 

analyses 


Description of equipment character- 
istics required by tasks or effect of 
equipment characteristics on task 
performance • 


( 4 ) 


QQPRI Data including: 






(a) Proficiency 


Skill characteristics which personnel 
should possess to perform the job 
sati sf actori ly * 




(b) Skill type 


Characteristics oi the job to be 
performed in terms of demands upon 
personnel. 




(c) Peisonnel 

a\ailability 


Definitions of AiFC type possessing 
necessary qualifications to perform 



the job, together with the probabil- 
ity of such personnel being available 
for the job . 



TABLE II (Continued) 



LIST AND DEFINITION OF 
HUMAN RESOURCES DATA INPUTS 



Item 

(5) Training requirements, 
including: 

(a) Anticipated training 
time 

(b) Required aptitude 



Definition 



Time needed to train to given level 
of proficiency. 

Job skills which training should 
provide. 



(6) Task analysis, including: 

(a) Task structure 

(b) Task criticality 

(c) Team performance 

(d) Probability of 
successful task 
completion 

( e) Task location 
(f) Task duration 



Task description in terms of function 
and equipment operated or maintained 
(See Item II (l)). 

Consequences of task being performed 
incorrectly or not at all. 

Number of perse nnel required to 
pei'form the task. 

Quantitative estimate of probability 
that the task will be completed success- 
fully by personnel (the converse, 
error probability, also is provided). 

Approximate physical area (e.g., flight 
line, shop) in which the task must be 
performed. 

Estimate of the time required to 
perform a task. 



(g) Difficulty index Estimate*! difficulty of task defined 

in terms of error probability and 
response time. 



(7) Time-line analysis, 

including task frequency 



Distribution over time, including 
overlaps, of individual task 
durations* 




*24 
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TABLE III 



STATEMENT OP WORK CONTENTS 

* 1. Description of system requirements 

* 2. Equipment requirements 

* 3. Top level flow diagrams 

4. Lists of personnel functions and tasks 

5# Maximum number of personnel or skill level requirements 

6. Task descriptions (eg, material contained in preliminary and 
full scale QQPRl) 

7. Personnel availability data 

8. Task characteristics 

9. Time line analysis 

10. Position descriptions 

11. Preliminary training requirements 



The asterisk (*) indicates what was included in the SCW for those 
subjects receiving HRD inputs incrementally. The remainder of the 
data were provided to these subjects progressively throughout the 
study. Other subjects received all SCW contents at the start of 
the study. 




l6 



design. For the comparison subjects (called the Incremental, 
group) only a minimal MR requirement was supplied in the SOW 
and HKD inputs were delayed until the third and fourth sessions, 
Ihe minimal MR requirement and the delay in HRD inputs were 
considered characteristic of present practice in system 
development • 

All equipment information was provided to ail subjects as 
part of the SOW. 

Ihble IV presents the sequence of test inputs and outputs 
for design of the experimental subsystem. 



3, Test Procedure 



The general procedure for the individual sessions was to 
determine the effect of a particular input on. the design task# 

At = start of each session, the engineer wa,s told his design 
tasA, the inputs available to him were described, and he was 
asked to review them (in the event he had not reviewed them since 
he was first handed them at the close of the previous session), 
The subject then performed his design task* 

About a half hour before the end of the session (unless he 
obviously was not finished, in which case the session would be 
continued to the following week), the subject was informed that 
his work was to be reviewed. His output then was reviewed by 
the investigator with him to elicit any additional information 
and particularly the reasons why particular design features were 
incorporated* At the same time, the subject war. questioned 
to determine whether: (l) he thought the input was useful, (2) 
the input was understandable and meaningful, (3) He used the 
input in deriving his design product, (4) the format of the 
input was satisfactory, ( 5 ) the timing of the input was 
appropriate, and (6) any additional information was needed. 

At the close of the session he was handed the inputs for 
the next session and asked to study them if he had sufficient 
time. 

The progressive development of the experimental subsystem 
was simulate! by scheduling each subject individually for a 
minimum of four weekly three-to-four-hour sessions (the length 
of the session depending on their speed) . For the Inca emental 
group this permitted the progressive inputting of HRD accoixling 
to the schedule described in Table IV, The subject had avail- 
able to him at each successive test session all the data 



TABLE IV 



SEQUENCE OF VEST INPUTS AND OUTPUTS 
FOR DESIGN OF THE EXPERIMENTAL SUBSYSTEM 



Introductory Session 



Session 1 



Inputs 

Omnibus Group: Complete S(JA 

Incremental Group: Partial SOW 

Output Required: Describe how maintenance will be accomplished 

and provide a detailed flow diagram of ground 
operations 



Session 2 



Inputs 

Omnibus Group: None 

Incremental Group: None 

Output Required: Identify elements of AGE required to perfoin the 

maintenance needed; provide functional descriptions 
of what each individual functional equipment is 
supposed to do. 



Session 



Inputs 

Omnibus Group: None 

Incremental Group: Third level equipment and personnel flow diagrams. 

Output Required: Supply equipment descriptions of individual AGE 

elements 
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TABLE IV (Continued) 



SEQUENCE OF TEST INPUTS AND OUTPUTS 
POR DISIGN OF THE EXPERIMENTAL SUBSYSTEM 



Session 4 



I nputs 



Oi in ib us Group: 
Incremental Group: 



Output Required: 



5sk analysis and QQPKL (position descriptions, 
training requirements and personnel availability 

statements) . . 

Complete detailed description of maintenance 

eauioment 




19 



(and his previous design outputs) from preceding sessions. At 
each session, the subject was asked to supply certain design 
outputs which the investigators hypothesized should be affected 
by the HRD input for that session. 



Determi nation of Design Outputs 
A. Types of Outputs 

ihe response secured from the subjects fell into two general 
classes, attitudinal or subjective outputs , and application, or 
product outputs . 

When an HRD input was first presented to a subject he was 
asked (after he had reviewed the input) to indicate his personal 
response to the input# By this is meant that the investigators 
sought to determine how the subject felt about his immediate 
input; whether he understood it, and if not, why; whether he 
felt he could use the input, and if not, why, etc# Since the 
engineer must first be positively motivated to accept an input 
before he applies it, subjective responses were secured before 
proceeding to more objective outputs. 

After the subject completed his subjective evaluation of 
the input, he was required to make use of the input by performing 
some engineering analysis or developing some eiigineering output, 
such as a drawing to which the HRD input was related# He was 
required to make use of the HRD input even though he may have 
indicated earlier that he could make little use of it. This 
was because his subjective response might or might not be 
related to his objective output. 



B. Subjective Outputs 

The kinds of subjective outputs to be sought of the subject 
were as follows: 

(1) Preference responses, e.g., I like/do not like the input. 

(2) Utility responses, e.g., I can/cannot apply the input 
to system design. 

(3) Knowledge responses, e.g., I u 1 ~rstand/do not under- 
stand the input. 



(4) Implication responses, e.g., I draw the following 
implications from the input; the following consequences 
result from the input, 

( 5 ) Schedule responses, e.g,, the input is too early/ too 
late/ just in time. 

( 6 ) Impact (effect) responses, e.g., my design is/is not 
influenced by the input. 

(7) Format responses, e.g., I would prefer the input to 
be In the following format. 

Although there was some slight overlap among these responses, 
each of these response types was considered separately because 
they could be combined in different ways, such as understanding 
an input but rejecting It as being inappropriately timed. 



C . Product Outputs 

Because the study was concerned only with the basic design 
concept, product outputs were largely of an anal ytic or decision - 
making type, eg* determination of functions, specification of 
equipment characteristics and operating modes. These were ex- 
pressed in terns of lists of functions to be performed by the 
system, equipment descriptions, equipment flow diagrams and 
procedures for operating the test equipment. 



Determination of Specific Measure s 

Measures of the effect of MR and SD inpits on subsystem 
design include the following: 

1 , Number and skill level of personnel estimated as required 
by subjects. 

P, Number and skill level of personnel required by subjects r 
designs • 

3. Number and types of manually operated equipment required 
by subjects. 

4. Number of automated equipments required. 

5. Number and type of special purpose equipment. 



6. Number of engineers in each group designing automated/ 
non- automated configurations* 

7* Number and type of design changes made after manpower 
requirements are changed. 

8. Number of controls and displays required by subjects. 

9- Number cf manual operations specified. 

10. Inter-subject variability* 

To understand the rationale for these measures, it is 
necessary to consider them in terms of the overall experimental 
design of the study. This study design and related measures are 
discussed in detail in the following section. 
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C. EXPEKQ1EN1AL DESIGN 



To explain the reason why the expe. " mental design for this study 
was created the way it was and the reasons for the various analyses 
which were performed, this subsection has been organized in terras of 
the specific study goals listed in Section I. 

1 , Will differences in amount and timing of HRD inputs result in 

different d es ign concepts? 

The basic hypothesis was that engineers receiving highly compre- 
hensive SCW's, including all the HRD inputs considered necessary for 
system design, would create design concepts different from those 
engineers who received incremental HRD inputs. 

To test this hypothesis the eight subjects were divided into two 
groups of four* The Omnibus group received aDl of its inputs, in- 
cluding HRD, prior to beginning design* It received no additional 
inputs throughout the remainder of the study. The Incremental group 
received prior to beginning design a basic SOW, including al l equipment 
inputs and, of the HRD inputs, only a minimally restrictive manning 
requirement* In sessions 3 and k the Incremental group received the 
remainder of its HRD inputs. 

Presumably, if the difference in amount and timing of the HRD 
inputs influenced design, it would be reflected in the basic design con- 
cept the engineer created. In addition to the actual design output for a 
system which would implement design requirements, subjects were asked 
to decide which of three concepts (manual, semi-automatic, or auto- 
matic operation) would best solve the design problem. The overall 
subsystem design could then be evaluated to determine in which of the 
three categories it belonged. Differences between the two groups 
could be evaluated by comparing the frequency of particular types of 
design concepts produced by subjects. Where changes in treatment 
conditions occurred (ie, in sessions 3 and 4), it is possible to 
determine what modifications, if any, we±e made to the design output 
of sessions 1 and 2. 

The effect of differences in amount and timing of HRD inputs 
could also be determined qualitatively by the following questions 
which were asked follow ig the first design session:* 



* A complete list of questions asked after each design session is. 
presented as paid; of Appendix I. 
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• Did. you have enough information in the SOW to develop tfie 
design concept? 

2. Enough equipment information? Enough personnel infojina^ion? 

3. Is this equipment information characteristic of SOW^; you 

work to? i 



k. 

5 . 

6 . 



Is this personnel information characteristic of SOW*s you 
work to? 

i 

What information that you did not have would you wish td have? 

Was the information in the SOW useful in helping you dec'ide 
upon your system configurations? Has enough information] been 
included in the personnel requirements statement? (For the 
Omnibus group only): What design implications would you] 

draw from the personnel requirements? | 



7* What information would you ordinarily have at the start j>f 
design? j 

| 

8, What items of information in the SOW particularly affect jed 
your design decisions? Why? j 

9- What was the effect of personnel requi remen ts on your de-sign 
concept? J 



At the conclusion of the second design session the foldovinh. 
relevant questions were asked: j 

I 

1* Have the equipment requirements acted in any way to constrain 
your design concept? If so, how? . 

2. Have the personnel requirements acted in any way to cor strain 

your design concept? If so, how? ! 

3. What equipment and personnel information which has not been 

provided to you would you wish to have? Why? I 



Individual session by session effects could be discerned by means 
of the responses to these questions. 

A word about the rationale for these questions. It was considered 
possible that because they received only the basic SCW, the Ircremental 
group might also feel that the minimally restrictive personnel 
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requirement did not provide very useful information* One might also 
find that the different personnel requirements affected the design 
concepts of the two groups differentially. 



2 . Will quantity constraints produce different effects on design 

than quality constraints? 

Manpower requirements are of two types: quantity (ie, number 

of personnel) and quality (ie, shill level). The previous study 
indicated that both of these requirements would exercise an effect 
only if they were formulated as constraints on design. Hie quantity 
requirement would therefore have to be phrased in the following 
manner: equipment must be designed so that no more than N personnel 

will be required to operate/maintain the equipment. The quality 
requirement would have to be phrased as: equipment must be designed 

so that it can be operated/maintained by personnel with a 3 (5/ or 7) 
-level skill capability. 

Two questions can be asked* One may ask whether one type of MR 
is more constraining than another* This question would be important 
in evaluating the relative emphasis to be placed on an MR in tradeoff 
decisions • 

One can also ask whether a detailed stringent MR has more effect 
on system design than one which is phrased in general terms only. 

In order to determine the differential effect of the two types 
of constraints, the two major groups (Omnibus and Incremental) were 
further subdivided into two helves. Two of the Omnibus and two of 
the Incremental subjects were asked to design to the quantity con- 
straint. Two of the Omnibus and two cf the Incremental subjects were 
asked to design to the quality constraints. The Omnibus subgrouping 
occurred in the first session; the Incremental subgrouping, in the 
third session. 

This further subdivision produced the following subgroups: 

0-N: Omnibus subgroup receiving only the personnel quantity 

constraint. 

The requirement levied on this subgroup was as follows: 

”5.1 Personnel Manning Requirements 

Use of AGE for the maintenance and testing of any indi- 
vidual system, subsystem or component of the missile and 
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associated missile-carrier aircraft shall not require more 
than two opera tors--one at or in the aircraft and one at the 
checkout or test equipment. " 

0-S: Omnibus subgroup receiving only the skill level con- 

straint. The requirement levied on them vas as follows: 

"5.1 Personnel Manning Requirements 

Equipment shall be designed to be operated by military 
technicians with an Air Force Speciality Code three-level 
skill only (see Appendix for definition of skill levels). 1 ’ 

Incremental Group 



The Incremental group was not subdivided in the first 
two sessions* They received and functioned for the first two 
sessions under the following minimal personnel requirements: 

u 5«l Personne l Manning Requirements 

Equipment design shall minimize the quantity and skill 
level of military personnel required to operate the equipment.' 1 



3. Will the imposition of MR constraints affect equipment char- 
a cteristics after the basic design concept has been developed ? 

He Air Force sometimes redefines its system requirements 
during the development of the system. The purpose of asking question 
(3) above is to determine the effect on design when such a reorien- 
tation occurs. Moreover, the effect of adding personnel requirements 
would help to indicate how influential MK constraints can be. 

In the first two design sessions the Omnibus group had received 
differential MR constraints. Half the group vas constrained to 
design to the 3-skill level (skill constraint or 0-S group). The 
other half vas constrained to design for a crew of two men (number 
constraint or 0-N group). To test the effect of imposing additional 
design requirements, the subgroup constrained by the skill require- 
ment (0-S) now also had to design for two men. The subgroup 
constrained to design for two men (O-U) now had also to design for 
a 3-skill level. 

The additional requirements were added jn the following manner: 
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"At this time the Air Force customer has decided to make his 
personnel requirements a bit more stringent than they were when you 
started your design. 

"(0-N Group) If you look at section 5»1 of your SOW which 
describes operator requirements, you will see the following state- 
ment: 



"Use of AGE for the maintenance and testing of any individual 
system, subsystem or component of the missile and associated 
missile-carrier aircraft shall not require more than two operators- - 
one at or in the aircraft and one at the checkout or test equipment » " 

"In order to minimize the skill level of the personnel needed 
to operate the maintenance ground equipment, the Air Force has 
levied the following additional requirement upon you: 

"Equipment shall be designed to be operated by military 
technicians with a three- skill level only." (Please refer to the 
definition of skill levels included in the Appendix to the SOW.) 

"(0-S Group) If you look at section of your SOW which 
describes operator requirements, you will see the following state- 
ment : 



"Equipment shall be designed to be operated by military tech- 
nicians with a three-skill level only." 

"In order to minimize the number cf personnel needed to operate 
the maintenance ground equipment, the Air Force has levied the 
following additional requirement upon you: 

"Use of AGE for the maintenance and testing of any individual 
system, subsystem or component of the missile and associated 
missile-carrier aircraft shall not require more than two operators-- 
one at or in the aircraft and one at the checkout or test equipment." 

"In this session we ask you to review the design concepts you 
created previously in the light of the additional requirements 
imposed upon you, and to make such changes as you feel would be 
necessary to bring your design in accordance with the added person- 
nel re qui r ement s . ,T 

At the conclusion of the session the following questions were 
asked: 



1. Did the additional personnel requirements make any difference 
to your design? If so, what changes did you make? 



2, What design implications did you draw from the added 
personnel requirements? 

3. Are these added requirements too stringent? Easy to handle? 

4* Was there enough information provided in the added i^equire- 
ment? 

5. What information would you wish included ‘r; he personnel 
requirements section of the SOW?” 

In the third session the Incremental group was subdivided in 
the same fashion as the Omnibus group in the first session* This 
produced the following subgroups: 

I-N: Incremental subgroup receiving only the personnel 

quantity constraint, 

1-S: Incremental subgroup receiving only the skill level 

constraint . 

These constraints were levied in the following manner: 

"In this session we are able to provide you with additional 
information secured from the Air Force, This information describes 
the personnel requirements to which you should design; in addition, 
the Air Force has made a number of analyses, included in an Appendix 
to the SOW, which describe what they think the inaintenance techni- 
cians in the system under design would be doing, 

,f Please replace section 5*1 of your SOW which describes 
operator requirements with the following statement: 

"(l-IJ Group) Use of AGE for the maintenance and testing of any 
individual system, subsystem or component of the missile and associated 
missile- carrier aircraft shall not require more than two operators- - 
one at or in the aircraft and one at the checkout or test equipment, 

"(l~S Group) Equipment shall be designed to operate by military 
technicians with a three-skill level only* ( Ihe definition of skill 
levels is included in Die Appendix.) 

n In this session vo ask you to review the design concepts you 
created previously in the light of the additional requirements and 
information now provided, and to make such changes as you feel would 
be necessary to bring your design in accord with the more stringent 
pe r s o nn e 1 re qui r em c n t r . ” 
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At the conclusion of the session the following questions (among 
others) vere asked: 

1 # Did the revised personnel requiremeirts make any difference 
to your design? If so, vhat changes did you make? 

2, What design implications would you draw from these require- 
ments? 

3, Are these requirements too stringent? Too easy? 

4, (I-N Group) What level of skilled personnel would you need 
to have to run your system under the personnel requirements 
imposed? Why? 

5* (i-S Group) Vhat number of personnel vould you need to have 
to run your system under the personnel requirements imposed? 
Why? 

The effect of imposing these requi rements on the subgroups could 
be tested by analyzing any changes made in subsystem design and also 
by responses to the questions asked following the design session*. 



h , Will the remova l of MR constraints affect equipment characteris- 
tics after the basic design concept has been developed? 

This question is parallel with that of question (3)« It MR had 
earlier constrained equipment design, then removal of the MH should 
lead to design changes. Such design changes vould provide additional 
evidence for the influence of MR on design. 

To study this problem all personnel constraints were removed 
from all subjects in the fourth design session. Instructions vere 
provided in the following manner: 

"Up to this point in time you have designed your systems to 
rather stringent personnel constraints. In this session ve vould 
like you to consider that all personnel constraints have been elim- 
inated, In other words, consider that you are able to design for 
an unlimited number of personnel and any skill level vh?ch you think 
you might need. Please review your design concepts from this stand- 
point, In the event that you restricted your designs to fit the 
personnel constraints, indicate what changes in your designs you 
vould wish to make, now that these restrictions have been voided. 
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"(For irx; omental groups only: To help you in your design, 
additional motion is now available from the Air Force describing 

its analyses of anticipated personnel characteristics and the tasks 
maintenance men would perform*)" 

After the engineer had completed his task, the following 
questions were asked: 

1. For incremental groups only: Did the additional information 

provided by the Air Force help any? Did it affect your 
design solution in any way? If so, in what way? If not, 
why not? 

2, Did lifting the personnel restrictions influence you in any 
way in changing your designs? In what way? If not, why 
not? 

3* We had made the assumption that the personnel requirements 
constrained your previous design in some ways. Is this 
true? How had these requirements affected your design? 

4 * Did your preferred design change any over the past four 

sessions when personnel requirements were changed? In what 
way? Why? If not, why not? 

5» Do you feel that these personnel requirements are realistic? 
Unrealistic? Would you rather not be constrained in this 
way? Why not? 

6. If you had to trade off personnel number and skill level, 
how would you do it? In other words, if you had a choice 
between fewer skilled technicians or more unskilled per- 
sonnel, which would you prefer? If you had more highly 
skilled technicians, could you use fewer people? If you 
had fewer people, would a higher skill level male up for 
the small size of the crew? 



5 • Which HKD inputs are preferred and utilized by engineers and at 
what stage in system development? 

Obviously it would be useful to determine which HRD inputs 
receive greater or lesser acceptance by engineers. Knowing this it 
would be possible to examine those inputs which were not being 
accepted in order to improve them. 

An answer to this question could be derived in two ways: (a) 

through the responses made by subjects to the questions asked 
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following each design session; and (b) through a paper and pencil 
test of HRD preference administered following completeion of the 
design. 

The following questions asked following the design sessions 
are pertinent to the problem: 

1. What design implications would you draw form the personnel 
flow diagrams; skill descriptions; from the QQPRI? 

2. What equipment and personnel information which has not 
been provided to you would you wish to have? Why? 

3* Are the skill level definitions understandable? 

4. Did the additional information provided by the Air Force 
help any? Did it affect your design solution in any way? 
If not, why not? 

To detennine at precisely what stage of system development 
HKD inputs were most useful a questionnaire test (HRD Inputs Test) 
was developed. Ibis is desciibed in Section III. 



6, How does the engineer^ concept of manpower for his system 

rela t e to his design and how does this concept relate to more 
objective methods of predicting manpower (eg. Barton et al.*s 
(1964) queuing model)? 

Spe APPENDIX III. 



Summary of Experimental Design 



The experimental design for the study can now be summarised in 
Table V which describes each of the two groups, the conditions unde:/ 
which they designed and the inputs provided to them. 

In sessions 1 and 2 comparisons are made between the Omnibus 
and Incremental groups to test the effect of different amounts of 
HRD information and different personnel requirements ( restrictive 
vs. non- restrictive) . 
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EXPERIMENTAL DESIGN FOR 
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In session 3 the comparisons are as follows: 

(1) Between the Omnibus group responses of session 3 and Omnibus 
group responses of sessions 1 and 2; to see if the added 
personnel requirements affect design responses; 

(2) Between O-N+S and O-S+N responses in session 3; to see whether 
adding different personnel requirements produces a differential 
effect on design responses; 

(3) Between the I-N and I-S subgroup responses of session 3 and 
the Inc remental responses of sessions 1 and 2; to see if tight- 
ening up personnel requirements and providing added personnel 
information will change design characteristics; 

(4) Between I-N and I-S responses in session 3> to study the effect 
of different personnel requirements. 

In session k the comparison is between session 4 Orr.inbus and Incre- 
mental responses and those of earlier sessions; to see if removing 
personnel constraints will affect existing designs. 

The experimental design associated different amounts of information 
with varying personnel requirements , so that the Incremental group not 
only had less HKD info mat ion than the Omnibus group, but also had a 
much weaker personnel requirement. As a consequence, it is impossible, 
except in a qualitative way, to differentiate the two conditions. Tie 
conditions were combined deliberately. Realistically, weak personnel 
requirements tend to accompany incremental itRD inputs. Although the 
reverse cannot be said to be true (i.e., that strong personnel require- 
ments are associated with earlier HRD, primarily because this situation 
is almost never found in actual procurement), the latter situation 
represents an ideal which the authors considered useful to contrast with 
the present situation. The point is not whether differences in personnel 
requirements are more important than up-to-date HRD, or what the effect 
of each is separately, but whether, as a total input package compre- 
hensive HRD plus strong MR will lead to more effective designs than the 
present system. What is being studied is a complex of factors, not 
single variables. 
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SECTION III 



RESULTS AND CONCLUSIONS 



A. INTRODUCTION 

Before proceeding to the specific study results, it may be helpful 
to the reader if he refers to Appendix II, (page H7) which presents 
some representative design outputs which subjects produced and which 
will give him a better ’'feel 1T for the nature of the design process. The 
following is a list of outputs illustrated in Appendix II: 

(1) Tradeoff Decision-Making Matrix; 

(2) System description; 

( 3 ) Flow diagram of system inputs and outputs; 

(4) Equipment operating procedure . 

Because of the complexity of the study results, we will summarize 
the study results before we proceed to detailed results, in which the 
results are categorized by the individual questions which the study 
sought to answer. 



TABLE VI 

SUMMARY TABLE OF RESULTS 



EXPERIMENTAL QUESTIONS 

1 . Will differences in amount and timing of HKD inputs result in 
different design concepts ? 



The answer is YES, as indicated below. 



Croup 

(4 designers 
each) 


Design Concept 


Manual 

Features 


No. of 
Test Sets 


Manpower Estimated by 
Subjects** 
Quantity Skill 


Omni bus- 

fall data 
in State- 
ment of 
Work) 


3 automatic 
1 semi-automatic 


166 


CM 

cu 


21 * 3-3 level 

10-5 level 
6-7 level 
2-9 level 


Incremental 
(data in 
stages) 


3 semi-automatic 
1 automatic 

. — n — ; tt - — 


77 


23.5 


16 

6-3 level 
6-5 level 
2-7 level 
G-9 level 



•x Ccru nine one ' on't know”, ** Includes both flight line and shop 
This question ii further discussed in paragraphs 1, 2, 3, 13 of the summary 



G c.f results. 




2 • Will quantity constraints produce different effects on design than 
quality constraints ? 

The answer is YES, as indicated below* 



Group 

(4 designers 
each) 


Design Concept 


Manual 

Features 


No. of 
Test Sets 


Manpower Estimated by 
Subjects *** 
Quantity Skill 


Quantity 

Constraint 


3 automatic 
1 semi-automatic 


103* 


16.5 


22 2-3 level 

11-5 level 
7-7 level 
2-9 level 


Skill 

Constraint 


3 semi-automatic 
1 automatic 


133 


29 


15** 7-3 level 

7-5 level 
1-7 level 
0-9 level 



* Incomplete data on one subject 
** Contains one "don’t know” 



*** Includes both flight line and shop. 

This question is discussed further in paragraphs 4 and 12 of the summary 
of results. 



3 . Will the imposition of MR constraints affect equipment characteristic s 
after the basic design concept has been developed ? 

The answer is YES, as indicated below. 



Group 


MR Constraint Added 


Change 
Ye s No 


Omnibus 


Quantity 


XX 




Quality 


X X 


Incremental 


Quantity 


X X 




Quality 


XX 



This quest i'. "j is farther discucrel in paragraphs 7 Mil ^ . 
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Will the removal of MR constraints affect equipment charac ^eristics 
after the basic design concept has been developed ? i 

The answer is NO. This question is further discussed in paragraph 9. 



5« Which HKD inputs are preferred and actually utilized by engineers 
and at what stage of development ? 

Most HRD information and requirements should be provided at the 
start of design if they are to be utilized. 

This question is further discussed in paragraphs 5> 10 and 11. 

6 . How does the engineer T s concept of manpower for his system relate to 
his design and how does this concept relate to more objective methods 
of predicting manpower ? 

This question is further discussed in paragraph 6 and Appendix III. 



B. SUMMARY OF RESULTS 

1. Analysis of the types of subsystems developed by subjects 
indicates a substantial difference between the Omnibus and Inci> cental 
groups. Three of the four Onnibus designs were automatic; thn e of the 
four j.cremental designs were semi-automatic. The difference ) ween 
the two groups fails to be statistically significant, primari'i r l . :ause 
of the small number of subjects involved. However, it would aj pc ar that 
the timing and amount of HRD plus the different personnel n [ 

had some influence on the type of subsystems developed. 

2. The Omnibus group also produced significantly (at Ihe .005 
level) more manual features in their designs than did the Incremental 
group. Differences between the two groups in terms of manpovr r required 
were also significant at the .08 level. There were no appreciable • 
differences between the groups with regard to the number of test s> ts 
developed. However, the number of test sets developed by sub 'eels 
ranged from 1 to 10, suggesting that one can expect in the nomal design 
situation considerable variability in design solutions. 

3« Six of the eight subjects (three in each group) reportc l that 
the personnel requirements imposed did affect their design deei; ions . 
However, as they saw the design situation, reliability, amount d work 
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required and time constraints were the primary factors in determining the 
type of subsystem developed. Personnel requirements o n their own vere 
not important enough to dictate design decisions, but did combine vith 
other factors to force the design concept adopted. 

4. Hie type of manpower requirement imposed also appeared to 
make some difference to subjects who vere differentially constrained by 
number and skill level. The number of test sets developed was substant- 
ially greater for subjects receiving the skill restriction (.17 level of 
significance). Skill restricted- subjects also produced more manual 
design features (at the .11 level). SkiU restricted subjects also 
required fewer personnel vith lover skill levels (significant at the .09 
level). The latter two points, in particular, suggest that the skill 
restriction may be more influential on design than the number restriction. 

5. As in previous studies, engineers selected their basic concept 
very rapidly. Ordy one of the subjects failed to establish his preferred 
design concept in the first session. 

6. Subjects experienced little difficulty in determining a level 
of manning which they considered appropriate to the subsystem they had 
designed. Omnibus subjects in general required a higher skill level and 
a larger crew than did Incremental subjects, a finding which suggests 
that the MR and HRD inputs provided this group had some influence on 
their manpower estimates. Engineers feel that a higher level of skill 
and more personnel are required in the shop, primarily because more 
manual., troubleshooting functions are performed in the shop. 

7. The additional HRD information provided to the Incremental 
group in the third session eithc^ resulted in a design change or would 
have assisted (had it been provided earlier) in developing the design 
concept. 

8. The additional MR constraints imposed on the Omnibus group in 
the third session did require engineers to make some changes in design 
concept. However, the fact that engineers resist making design changes 
after their design has been formalized (also found to be true in previous 
studies) tended to reduce the extent of changes demanded by the new MR 
constraints. Much the same effect was found for the Incremental group. 

9. Removal of the MR constraints did not in six cases out of 
seven cause changes in design concepts, although some subjects noted 
that if the MR constraints had originally not existed, theiz' design 
would have been somewhat different. This result is entirely in accord 
vith earlier studies which suggest the relative inflexibility of design 
concepts once they are formalized. 
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10# All subjects indicated that they would prefer most HRD infor- 
mation and requirements to be provided at the start of design# Failure 
to supply this information leads to some lack of clarity for engineer 
and non-utility of this information when provided later in design. The 
following items of personnel information are particularly desired as 
soon as possible: 

(1) Maximum number of personnel for which equipment is to be 
designed; 

(2) Description of jobs personnel will have to perform; 

( 3) Personnel skill level; 

(4) Number and type of personnel to be made available to run the 
system# 

11# Although engineers desire as much information about the system 
as possible, and although this information does affect their design 
concept, they have difficulty verbalizing design implications from the 
information# This suggests that it is necessary for the human resources 
specialist to specify for the engineer in the SOW the design implications 
of the HRD the former provides. 

12# The engineer subjects in this study almost unanimously felt 
that skill was of greater significance to the operation of the system 
than the number of personnel available# Skill can compensate for lack 
of personnel, but additional personnel cannot compensate for lack of 
skill# 

13* In general, the results of this study verify [-he hypotheses 
advanced as an output of the previous study: that amount, timing and 
nature of the personnel information supplied to design engineers will 
exercise some effect on the design concept# 
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C . DETAILED RESULTS 

The effect of amount and timing of HRD inputs on design are 
summarized in Table VII and. are described in detail in the 
following pages. 



r 

TABLE VII 

THE EFFECT OF AMOUNT AND TIMING OF HUMAN 
RESOURCES DATA INPUTS ON DESIGN 



Design 
Group 
(4 each) 


Design 
Concept * 


Average 
No. of** 
Manual 
Features 


Average 
No . of 
Test 
Sets 


Ave rage 
Re: 

Quantity 


; Manpower 
^ui red 

Skill Value 


Omnibus 
(all data 
in SOW) 


3 automatic 
1 semi -auto. 


50-3 


5.5 


5.5 


38.5 


Incremental 
(data in 
stages) 


1 automatic 
3 semi -auto. 


17.5 


6.0 


6.0 


42.5 



* 



difference significant at 



.23 level 



** 



difference signiii iant at 



.005 level. 
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1 . Will differences in amount and timing of KRD inputs result in 

different design concepts? 

The answer to this question is determined by the nature of the 
system designs produced by subjects of the two groups • Subjects 
were asked at the start of the study to make tradeoff decisions 
among an automatic , semi-automatic and manual design. 

In accordance with what has been learned previously concerning 
the speed with which basic design concepts are developed by engineers, 
(and the propensity for automatizing equipment) all subjects immedi- 
ately rejected the manual design possibility. The reasons foi 
rejecting the manual alternative were the reliability, maintain- 
ability and turn-around requirements. 

Because of the rejection of the manual design alternative, the 
essential comparison is between the numbers of subjects selecting an 
automatic vs. those selecting a semi-automatic solution. 

In determining which design concept was utilized, the investi- 
gators allowed subjects to characterize their designs; that is, 
subjects assigned the semi-automatic or automatic description to 
their own designs. These categorizations were later checkea by the 
investigators against the actual subsystem design produced. 

An automatic system was defined by subjects as a system in which 
the mainten nee man merely initiated machine sequencing and observed 
the results. In a semi-automatic system the maintenance man initiated 
machine sequencing, but the machine ran only to a predetermined stage 
in its operation, after which the technician had to decide whether 
to continue the sequencing and, if there were alternative uests that 
could be made, to decide which test to run next. 

If there were no consistent tendency on the part of subjects to 
select either a semi-automatic or an automatic design solution, one 
would expect the frequency of types of solution to be; (l) equal 
between the two groups (ie. the same numbers of automatic and semi- 
automatic solutions in both groups); (2) equally divided between 
semi-automatic and automatic (ie, each group would have two automatic 
and two semi-automatic solutions) . 
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The design solutions for the individual subjects, together 
with the number of test sets they required for these solutions, 
are shown in Table VIII. 



TABLE VIII 

SUMMARY OF DESIGN SOLUTIONS 



Sub j ect 


Group 


System Type 


Number of 
Test Sets 


Y 


I-S 


Semi -automatic 


4 


L 


I-S 


Serai -automatic 


10 


F 


I-N 


Automatic 


6 


Me 


I-N 


Serai -automatic 


3-4 


Mo 


0-S 


Semi -automatic 


5 


Ma 


o-s 


Automatic 


10 


K 


0-N 


Automatic 


1 


V 


0-N 


Automatic 


6 
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When the actual frequencies of design solutions are compared 
statistically with the expected frequencies using the Fisher Exact 
Probability Test ( Siegel , 1956 ), the result is a probability value 
of , 23 * The Usher test, to quote the author, 

"is an extremely useful nonparametric technique for analyzing 
discrete data.... when the tvo independent samples are small 
in size. It is used when the scores from tvo independent 
random samples all fall into one or the other of tvo mutually 
exclusive classes. In other words, every subject in both 
groups obtains one of tvo possible scores . . . .The test deter- 
mines whether the tvo groups differ in the proportion with 
which they fall into the tvo classifications..." (p. 91 ). 

Although the .23 value is insufficient to reject the hypothesis 
that the system designs are the result of individual variability in 
subjects, it is necessary to qualify the answer somewhat. It should 
be noted that the tvo groups produced exactly reversed solutions, 
which suggests a non-random factor. Unfortunately, the statistical 
techniques available for making comparisons of discrete frequencies 
do not take into account the difference in direction of response 
between the tvo groups. 

In considering the meaning of the statistical results, the 
following mast be kept in mind. The differential treatments 
involving HRD inputs were only one factor dete mining the choice 
of a design solution. The primary factors affecting the design 
concept were the reliability, maintainability and time requirements, 
with the MR and HRD inputs providing only an increment to these 
factors. Verbally subjects indicated that MR/HRD inputs did 
influence their design decisions. Since the primary factors affecting 
the design concept will always be the equipment requirement, it must 
be considered that, to the extent that they reflect the influence of 
personnel inputs, the differences in the frequency of experimental 
design solutions are in reality very indicative. The point is that 
one cannot expect personnel requirements and inputs in and of them- 
selves to determine design responses. To the extent that HRD inputs 
exercise any impact, on design responses (as they did in this study), 
their effect was as significant as one could reasonably expect then 
to be. 



Assuming thcat the differences in response frequency between the 
two groups are not merely random, it is necessary to explain them. 
Why should the Omnibus gi'oup have produced more automatic and the 
Incremental group more semi-automatic responses? Since the Omni! us 
group received tore restrictive Mv constraints, it would seem 
reasonable that t r.ese constraints would strengthen the subjects T 



tendency to automatize their equipment. The fact that they were 
required to design for tvo men or an Air Force Speciality Code 
(AE3C) of three-level would cause them to throw the largest part 
of the burden of system operation on the equipment. Both the 
number and skill constraints would have suggested the desirability 
of reducing the influence of the human in the subsystem. 

On the other hand, the minimal MR constraint received by the 
Incremental group would have reduced ( to a certain extent) the 
tendency to automatize their designs. 

What evidence is there for these hypotheses, based on the 
subjects 1 verbal responses? 

Six of the eight subjects reported that the MR constraints did 
affect their design solutions. The consensus of responses can be 
summed up as follows: the tasks required and the personnel constraints 
imposed, when combined with the time requirements , seemed almost to 
dictate a particular level of automatic ity. The level of personnel 
to a certain extent constrained the level of automaticity selected. 

One characteristic of design which never fails to surprise is 
the great degree of variability f our d in the more detailed aspects 
of the design solutions selected to answer bhe same requirements. 

This can be seen by referring again to Tb.ble VI. The number of 
individual test sets ranged from one to ten. Unfortunately, the 
number selected did not differentiate between the Omnibus and Incre- 
mental groups; both groups had a very similar range. 

Other indices of variability: One subject packaged all his 
test sets in a single cart; another had individual test sets which 
had to be hooked up with cabling before the tests could be run. 

Some test sets were individually packaged; others were individually 
packaged but placed in a '/an or on a cart. 

The implication of this variability for the writer of design 
specifications or procurement documents is that as much detail as 
possible is needed to restrict this variability. 3ir.ce design is 
so variable, the procuring agency cannot be sure of getting the 
design it wishes unless it attempts to restrict this variability 
very severely by specifying as clearly as possible what it expects 
from the designer. 

With regard to the amount and type of information provided in 
the SOW on the basis of which the designs were developed, the fal- 
lowing was reported: 
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Half the subjects stated that the level of information provided 
in the SOW was very similar to that they are ordinarily given to 
work with, while the other half indicated that it was much more 
complete than they were accustomed to. Apparently there is great 
variability also in the format and detail of existing procurement 
specifications to which engineers must design. 

Subject responses indicated that they hardly, if ever, received 
any data regarding personnel. In some cases they have to generate 
the information themselves. While it cannot be said that the personnel 
information was of major significance in affecting design decisions, 
for certain subjects it was helpful. One subject found the descrip- 
tive material in the QQPRI to be helpful in developing operational, 
procedures for his design. Another reported that ’personnel had a 
great deal to do with how I design” and so presumably the information 
provided was of use to him. 

The fact that most of the subjects reported that the HRD inputs 
did not significantly affect their designs must be considered in the 
light of their responses to the equipment information, which they 
also said had little effect, except for system and checkout com- 
plexity and required turn around time. The point is that except for 
explicitly stated requirements engineers do not really know (or at 
least cannot verbalize) what items of information (either equipment 
or personnel) really influence their design. They are unable to 
indicate what items of information, except for minor specific 
details, they would like to have ■’n order to begin designing. 

Another measure which was applied to the subjects 1 design 
outputs also indicated significant differences between the two 
groups. It will be recalled that among the measures to be applied 
to the design outputs (see Section II) were number and types of 
manually operated equipment, number of controls and displays 
required and number of manual operations specified. Since the 
number of instances of each of these measures was fairly small, it 
was considered desirable to combine all of these instances into a 
single measure. The results are shown in Table VII, A ,T t' r -test 
indicates that the difference between the two groups is statistically 
significant at the ,005 level, even though one of the subjects of 
the Omnibus group had incomplete data. 

It may appear strange that the Omnibus group, which produced 
most of the automatic designs, should also have indicated signifi- 
cantly more manual design features ( automatic ity and manual features 
being somewhat inconsistent). This is not a contradiction, but 
rather a reflection of what appears to be a greater awareness on 
the pai't of the Omnibus group of the need to consider operator 



features, (This greater awareness might have been predicted as a 
result of the presentation of all HKD inputs - including stringent 
manpower requirements- at the start of design, which presumably 
alerted these subjects to the "customer’s" interest in operator 
factors • ) 

As a consequence, Omnibus subjects were more painstaking about 
detailing the manual characteristics they needed for their systems, 
even though they may have needed fever of them. The Incremental 
subjects, on the other hand, did not describe in as much detail the 
manual characteristics implicit in their designs. Hence the dif- 
ferences reflected in Table IX indicate a difference in sensitivity 
to operator factors, rather than a difference in design. This added 
sensitivity is of course essential to secure adequate consideration 
of personnel requirements, and is the reason why elsewhere the 
authors have suggested that the presentation of as much HRD as 
possible (whether or not these serve to constrain design) is useful. 

Differences between the Omnibus and Incremental groups are not 
significant when one considers the number of test sets required by 
subjects (see Table VII ) . The mean number of test sets for the 
Incremental group is 6.0, while the mean number for the Omnibus 
group is However, there are substantial differences between 

the individual subgroups which will be discussed under question (2). 

Another measure referred to in Section II dealt with the number 
and skill level of personnel estimated as required by subjects. 

These will be analyzed in greater detail under question (6), which 
asks how the engineer’s manpower concept relates to his design; but 
it is interesting to see whether the groups receiving different 
amounts of HRD at the start of design also differ in terms of the 
manpower they require. 

For this analysis a manpower rating value was determined for 
each subject. This was secured simply by multiplying the number of 
personnel the subject indicated that he required times the skill 
level of those personnel. For example, if a subject said he needed 
two 5-level and one 7-level personnel to run his subsystem, he would 
receive a value ofl7(2x5 + lxT~ 17)* Although this method of 
quantifying manpower is highly arbitrary, it does serve to illustrate 
major differences between the two groups. 

Table X lists the manpower values for each subject. When 
the Kruskal-Vallis One -V ay Analysis of Variance test (Siegel, 1956) 
was applied to the data, the results are statistically significant 
at about the ,0G level (ll = 3.5; H - 3.84 for .05 level). The 
results suggest that the two groups are indeed differentiated in 
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TABLE IX 



SUMMARY OF MANUAL DESIGN FEATURES 



Subject 


Group 


System Type 


Number of 
Manual References 


Y 


I-S 


Semi-automatic 


17 


L 


I-S 


Semi-automatic 


15 


F 


I-N 


Automatic 


25 


Me 


I-N 


Semi-Automatic 


13 








Total = 70, M = 17.5 


Mo 


0-S 


Semi-automatic 


^3 


Ma 


o-s 


Automatic 


58 


K 


0-N 


Automatic 


50 


W 


0-N 


Automatic 


* 








Total - 151 , M = 50.3 



* Incomplete Data 
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TABLE X 



SUBJECTS ESTIMATES OF MANPOWER REQUIRED 
(FLIGHT LINE AND SHOP) 



Subject 


Group 


System Type 


Manpower Value * 


Y 


I-S 


Semi-automatic 


18 


L 


I-S 


Semi-automatic 


12 


F 


I-N 


Automatic 


20 


Me 


I-N 


Semi-automatic 


22 

Total - 72, M = 18.0 


Mo 


O-S 


Semi-automatic 


23 


Ma 


o-s 


Automatic 


20 


K 


0-N 


Automatic 


52 


* W 


O-N 


Automatic 


. 34 

Total = 129, M - 32.2 

.j 


* Manpower 


value = Number 


x skill level. 
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terms of the manpower they require. Again the Omnibus group, which 
received all its HKD inputs at the start of design, required sub- 
stantially more and higher shilled personnel. One can hypothesize 
that the greater amount of HKD information provided to the Omnibus 
group at the staid; of design emphasized the importance of manpower 
to the subjects of that group, and caused them to demand a larger 
number of skilled personnel (despite the fact that most of their 
subsystems were automatic) . 

The preceding analysis (Table X) was based on the subjects' 
own estimations of the manpower they felt they needed for their 
systems. In another analysis each design concept was analyzed by 
a human factors specialist not on the study team* and the number 
and skill level of personnel which that design concept would have 
required was determined. In other words, this manpower analysis 
was independent of the subjects 1 own estimates. Table XI shows the 
results of that analysis for the shop activity (a3_l flight line 
designs required the same number and skill level, one 7"l^vel, one 
9-level) . 

Manpower required by Ctanibus group designs was somewhat, al though 
not significantly, less than that required by Incremental group 
designs, according to the Randomization Test for Independent Samples 
(Siegel, 1956) . This is in accordance with the hypothesis that 
extensive HKD information plus stringent manpower requirements 
supplied at the start of design should result in more efficient 
design (from the standpoint of MR requirements) chan incremental 
inputs and minimal manpower requirements. 



* The authors are grateful to Mr. E. A. Thompson of the Bunker-Ramo 
Human Factors Department, for performing this analysis. 
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TABI.E XI 



HUMAN FACTOR SPECIALISTS ANALYSIS OF 
DIFFERENCES IN MANPOWER REQUIRED BY SUBJECT DESIGNS 



Subject 


Group 


System Type 


Number (N) Skill (R) 

Required Level Req'd 


Manpower Value 
(N x S) 


Y 


I-S 


Semi-automatic 


1 x 5-level 
3 x 7-level 


26 


L 


I-S 


Semi-automatic 


2 x 7- level 
Xx g-level 


23 


F 


I-N 


Automatic 


2 x 5-level 
3 x 7-level 


31 


Me 


I-N 


Semi -automatic 


1 x 5-level 
3 x 7-level 


c6 

M = 26,5 


Mo 


0-S 


Semi-automatic 


1 x 5-level 
4 x 7-level 


33 


Ma 


0-S 


Automatic 


1 x 5-level 
2 x 7-level 


19 


K 


O-N 


Automatic 


2 x 5-leveX *= 


10 


;w 


0-N 


Automatic 


4x7-leve3 = 


28 










M = 22.5 



Note: Table represents analysis for shop activity only; all flight 

lino displays require 1 the same number and skill level, one 
7-level, one 9 -level* 
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2 . Will quantity const ra . ii its proauce different effects on desig n 

t han quality constraints? 

Question (2) asks whether there are differences between the 
subgroups, regardless of the major group treatments* In other words, 
regardless of the amount and timing of IIHD inputs , are there dif- 
ferences between subjects receiving the skill restriction (design 
only for 3-level personnel) and those receiving the quantity 
restriction (design foi" a maximum of two men)* A comparison must 
therefore be made between X-S and Q~S subjects on the one hand and 
I-N and 0-N subjects on the other* To make these comparisons the 
reader should refer to Tables VII, VIII and IX. 

In Table V3T the measure of interest is the number of test sets 
required* In both Omnibus and Incremental groups the mean number of 
test sets required by subjects receiving the skill restriction is 
substantially greater than that required by subjects receiving the 
number or quantity restriction* However, this difference is 
statistically significant at the *17 level only (using the Randomi- 
zation test for two independent samples (Siegel, 19^6) . 

In Table V3XI, comparing the manual design features included in 
subjects' designs, the differences between the sub-groups appears 
to be submerged by the over-whelming effect of the differences in 
amount and timing of HFD information provided. In any event, the 
fact that only incomplete data are available for one of the subjects 
of the Omnibus group makes a statistical comparison highly tenuous* 
Nevertheless, ths differences are statistically significant at the 
.11 level (Randomization test for two independent samples (Siegel, 

1956) . 



In Table IX, differences in manpower required by subjects to 
run the subsystems designed, major differences are found between the 
groups constrained by skill and those constrained by personnel 
number. Skill-restricted engineers tended to require fever and less 
highly skilled personnel . This is in accordance with the manpower 
requirements imposed on these subjects. The differences are statis- 
tically significant at the .09 level (Randomization test for two 
independent samples (Siegel, 1956)). 

There appears then to be sone quantitative evidence that types 
of MR constraint produce differential effects on design responses, 
but further investigation is required if a definitive answers is 
to be given to this question. 

Qualitative data a re also available from responses to question 
number 6 in the fourth session. Oliis question asked subjects 
generally how they would trade off skill against quantity of personnel. 
The responses given were as follows: 



SO 



Y: Skill level can compensate for numbers, but not vice versa* 

L: Must be designed for the lowest level of personnel, but 

raising skill 3.evel does not compensate for lack of 
personnel in operational systems, 

F: Would prefer to see systems operated by a lesser number 

of skilled people and believes that skill can compensate 
for number. 

Me; Fewer skilled people are "greatly preferable" to numerically 
more unskilled people,, . . Too many unskilled people are a 
burden on the skilled people because they must supervise 
and train the unskilled. 

Mo: Similar to Y’s response above. 

Ma: fckill is the most important factor. 

K: "How much work do I have to do is how many hands I need” 

is the factor which allows him to arrive at how many 
’'hands 1 ' he would need because of sinultaneoas jobs. Then 
"you look at whether you can reduce the number of hands 
with higher skill levels or design changes. tT 

W: Skill can replace number. 



It would appear then that although engineers do not view skill 
and number as a black and white dichotomy, most of them assign a 
higher priority to skill; and, faced with a quantity/ quality tre&eoff, 
would almost always opt for higher quality. 



3* Will the imposition of MR co nstraints affect equipment charac- 
teristics after the basic design c o ncept h a s been developed? 

The responses made by subjects to the manpower requirements 
added in session 3 indicated quite positive3_y that these requirements 
did or would influence design. Table XII indicates that six of the 
eight subjects would change their designs in various ways. However, 
the influence of these added requirements was only moderate, because 
of the well known resistance of engineers to modifying their design 
once it has been fully conceptualized. The results of session 3 
indicate once more the potential influence of stringent personnel 
requirements ort the design process. 




CO 



51 



TABLE XII 
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The following characteristic responses were made: "I might be 

able to simplify tny design somewhat"; "would probably give more 
attention to automating the system than I did originally", because 
of "number of functions to be performed" and "limited time to be 
performed in"; "system would change to be a bit more manually 
oriented" (in order to accomplish the continuity testiig he would 
simplify his checkout equipment); "would add more sophistication 
to my equipment"; "would reduce the amount of information fed back 
to the technician on the flight line and place more emphasis on the 
shop personnel". 

There appear to be no significant inter-group differences in 
effect of adding MR constraints. 



h. Will the removal of MR constraints affect equipment character- 
istics after the basic design concept has been developed? 

In general, the removal of the MR constraints once the system 
had been designed to personnel restrictions did not cause the engineer 
to modify his design. Six of the seven subjects responding (one 
refused to reply) indicated that lifting the MR restrictions did not 
influence their design. One subject reported that he could now "get 
by with a simpler system with lover cost," etc. The system would 
be d ore manual, ie, "more buttons to push". He could proceed to a 
lower level of maintenance and component replacement. 

5* Which HKD inputs are preferred and actually utilized by engineers 
and at what stage of development ? 

It is obvious from the results discussed previously that MR 
constraints, viewed as a type of RRD input, are utilized by engineers 
and do influence their design to a certain extent. Is this true of 
other HRD inputs? 

Two sources of data are available relative to this question. At 
the conclusion of the design period a number ol paper and pencil 
tests were administered to subjects (described in Meister et al. , 

196 ?) j one of which (HRD Inputs Test) sought to determine at what 
stage of system development various HRD inputs would be most accept- 
able to engineers. The results of that test, expressed in terms of 
frequency of engineers preferring to receive inputs at various times, 
are shown in Table XIII. 

The technique was applied to determine whether the distri- 
bution of subject, responses among the four time periods varied 
significantly from chance. The distribution of six of the data 
items was significant at the .05 level or better. 
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TABLE XIII 



PREFERENCE POR HRD INPUTS AT SYSTEM DEVELOPMENT STAGES 



In RFP 
or 

Initial During 

Information SOW Predes5gn 


At Start 
of 

Detail 

Design 


During 

Detail 

Design 


X 2 


P 


1. Maximum number of person- 
nel for which equipment is 
to be designed. 


7 


1 






17.0 


.001 


2. Description of jobs 
personnel will have to 
perform. 


6 


2 






12.0 


.01 


3# Personnel skill level. 


6 


2 






12.0 


.01 


4. Type and length of train- 
ing personnel will have. 


4 


3 




1 


5.0 


NS 


5* Amount of experience 
personnel will have. 


4 


3 


1 




5.0 


IIS 


6. Duration of each 
personnel job. 


5 


2 




1 


7.0 


.10 


7* Number of personnel to 
be made available to run 
system. 


6 


1 




1 


11.0 


.02 


8. Type of personnel to be 
made available to run 
system. 


6 


1 




1 


n.o 


.02 


9. Cost of training personnel. 


3 


1 




1 


3*5 


NS 


10. Manpower life cycle cost. 


4 


1 




1 


5.0 


NS 


11. Probability that personnel 
will make certain kinis of 
errors . 


2 


6 






12.0 


.01 


12* Equipment characteristics 
required by personnel charac- 
teristics or tasks. 


4 


3 


1 




5.0 


NS 


13* Criticality of tasks per- 
formed by personnel. 


5 


2 




1 


7.0 


.10 


14. Difficulty level of tasks. 


1 


2 


1 


1 


3.0 


NS 
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For those items which reflect little consistency on the part 
of subjects and are relegated to later stages of system development 
{ie, cost of training personnel, manpower life cycle cost, difficulty 
level of tasks), subjects indicated that these items are of little 
importance in the performance of their design tasks; hence they are 
not concerned with receiving this information immediately. Indeed, 
certain subjects refused to respond to a number of items on the 
ground that the item was an Air Force responsibility and not a 
contractor responsibility. 

The following items of personnel information are, however, 
desired as soon as possible: 

(1) Maximum number of personnel for which equipment is to be 
designed; 

(2) Description of jobs personnel will have to perform; 

( 3) Personnel skill level; 

(4) Number of personnel to be made available to run the system; 

(5) T^pe of personnel to be made available to run the system. 

In general it can said that engineers prefer to receive as 
much ILRJj information as possible. Corroborating data can be found 
in the verbal responses of Incremental subjects in the third session. 
Two reported that they do not ordinarily receive this type of infor- 
mation; of the other two, one said that the ordinarily expects to 
have to "generate” this kind of information on his own, while the 
second said he "would almost have to receive this kind of information 
to do the job decently." 

Another individual in session 3 indicated that presentation of 
HRD information earlier in the design time frame would have allowed 
him to reach his design decisions earlier and easier. 

It is important to note that engineers respond primarily to 
inputs phrased as design requirements, and hence, Vito the exception 
of MR, HRD inpuos held little value to them. At the same time 
engineers want to see as much information as possible, so they can 
pick and choose whatever they wish from it. 

Ir. general, even when the engineer says that he does not use 
personnel inputs, they create a context or bias toward one kind of 
solution or another, if one wishes, then, to have engineers pay 
more attention to personnel factors, it is desirable to provide them 
with considerable HRD information. 




55 



6 • How does the engineer's concept of manpower for his system 

relate t o his design and how docs this concept relat e t o more 

ob5ective methods of predicting manpow er? 

It was possible to secure manning estimates from all but one 
subject as the conclusion of session 1. These are shown in Table XII 
below, together with revisions made during tie course of their 
design. F stands for flight line, S for chop. 

Examination of Table XIV reveals some Interesting differences 
between the Onibus and Incremental groups. The Incremental group 
unanimously postulated two men for the flight line and two for the 
shop. These were at the 3 or 5 level (with only one exception, a 
7-level for the shop) . Changes In Incremental group manning as a 
result of new HRD inputs were minor. 

In contrast the Omnibus grouj> postulated sane what higher numbers 
of men required, particularly for the shop. A comparison of Incre- 
mental vs. Omnibus estimates for shop manpower (quantity) alone, 
using the X 2 technique, reveals that the difference is significant 
at the .06 level. Skill levels were also somewhat higher. The larger 
manning required by Omnibus subjects is attributed to their greater 
sensitivity to the ’'customer's" interest in personnel factors 
(resulting from the large number of HRD inputs presented at the 3tart 
of design). The distribution of (revised) skill levels, broken out 
by group and by type of system designed, is shown In Table XV. . 

The following conclusions appear warranted: 

1. Engineers can develop estimates of the manning needed for 
their systems, but in a number of cases these estimates are not very 
precise. It is apparent that they need expert help in developing 
these estimates. 

2. Engineers feel that a higher level of skill and more personnel 
are required in the shop situation, primarily because they conceived 
more manual, troubleshooting functions being performed in the shop. 

3. Manpower estimates, as the engineer sees them, seem to be 
more highly related to the type of subsystem he designs than to any 
imposed personnel constraint based on questions following design 
sessions. Flight line estimates of skill varied significa ltly (at 
the .05 level) from what was required by the skill constraint. 

One would expect the automatic subsystems would elicit estimates 
of fever personnel or lover skill level, but this hypothesis was not 
reflected in subject estimates. For example, there is a elcarcut 
difference between skill .levels predicted for semi-automatic and 
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MANPOWER ESTIMATED 3Y SUBJECTS FOR EXPERIMENTAL SUBSYSTEMS 
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Note: Flight, line manning remained constant at l-(7-leve^) , l-(9-levei) . 



automatic subsystem designs with automatic designs requiring higher 
skill personnel. This is particularly marked at the shop level, 
where the distribution of skill levels estimated by subjects varied 
significantly (at the .02 level) from what would have been expected 
on the basis of the skill constraints. 

These results are similar to those found in the study by 
Meister et- al. , 1968, in which a vide discrepancy between type of 
system designed and manpower requirements was also found . The 
question can therefore be raised as it was in the previous stud;/ 
whether designer estimates of required manpower are realistic, ;i 
despite engineers 1 insistence that estimated manning is actually 
essential to operation of their systems. ' 

4 . It is noteworthy that the MR limiting manpower to two was 
violated in several instances, the designers either ignoring it ' or 
classing It as unrealistic although in general it was compiled with. 

An unanswered question is whether the timber restriction to two men 
was actually effective in those cases in which this number was rot 
violated, or whether the nature of the system designed was such that 
for these system designs only two personnel were needed. Enginreis 
feel that they are responsive only to the nature of the systems; they 
design, although other evidence (discussed, under question (2)) suggests 
that they are unconsciously influenced by MR. 

j 

It is interesting to note also that compliance with the personnel 
quantity restriction was far greater than compliance with the skill 
quality restriction. This may be .related to the engineers 1 welt 
known difficulty in understanding the meaning of the skill requ.re- 
rnents • Certainly skill estimates were more difficult for engin iers 
to develop than were the number estimates. 

5 . The skill level requirement restricted manning to 3 “ 1 -vel 
personnel. It is significant that almost all subjects violate! this 
restriction. If one compares the skill levels estimated by subjects 
who were skill restricted with the skill levels estimated by subjects 
with the quantity restriction, there is some evidence (of a tentative 
nature only, of course) that subjects on whom the skill restriction 
was imposed had fever higher level skills even when they violated the 
requirement. Prom that standpoint it can be said that the skxlL 
level restriction was somewhat effective. 

6. An independent estimate of the manning required for tiis 
subsystem was performed by a Bunker- Rarao human factors specialist* 
not involved in the project and using only the basic SOW (excluling 



* Mr. E. A. Thompson of the Bunker-Ramo Human fhetor* Department 
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TABLE XV 



DISTRIBUTION OF ESTIMATED MANPOWER BY GROUP, 
WJNTS NANCE AREA ALT) SUBSYSTEM TYPE 



Skill Lav el by Group 

Flight Line Skills Shop Skills 

3 5 7 9 Total 3 5 7 9 



I-S 


i 

2 


f 

2 






4 


V 


1 






I-N 


1 


3 






4 




2 


2 




0-3 


2 


2 






4 




2 


1 




0-N 


1 


2 


1 




4 




3 


4 


P < 
' ] 


Total 


6 


9 


1 


16 


3 


8 


7 


2 



Total 

L 

h 

o 

O 

s 



20 



Skill Level b y System Type 
Flight Line 



Shop 





3 


? 


T 


9 


Total 


3 


5 


7 


9 


Semi- 

Auto. 


5 


3 






g Semi- 
Auto. 


3 


3 


3 




Auto, 


1 


7 


1 




^ Auto.* 




5 


4 


2 


Tota.t 


6 


10 


1 




17 


3 


8 


7 


2 



Total 

9 

n 

20 



* Includes one "Don't kn~w 11 response. 
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any personnel requirements and information) as the basis for his 
estimates. Manning was estimated for both automatic and semi- 
automatic configurations, as follows: 

Automatic Semi -Automat i o 



Flight Line * 


2 men, both 


Flight Line 


- 2 men ; one 5 j one 




5-level 




7 -level 


Shop - 


2 man, both 
7-level 


Shop - 


2 men, both 7-level 



It should be noted that the skill levels estimated by the human 
factors specialist agree far more with Omnibus group estimates than 
they do with Incremental estimates (see Table XV i) . If we tare the 
independent estimate as the "true' 1 manpower required, by these sub- 
systems, the violation of the skill constraint by the Omnibus group 
is now much more understandable. If we assume the < stimates made 
by the independent hitman factors specialist are morn realistic than 
that permitted by the skill constraint, then the Omnibus group 
subjects were realistic iu violating the skill constraints imposed 
on them. It is possible that in violating the skill constraint to 
a greater extent than the Incremental group, the Ora ii bus group was 
reacting to the additional HRD information they possessed, which 
suggested ai increased task complexity. 

TABLE XVI 



COMPARISON OF THE DESIGN ENGINEERS ESTIW.il. 

OF SUBSYSTEM MANPOWER WITH HUMAN FACTORS ANALYSIS 
OF SUBSYSTEM MANPOWER REQUIFEMENTS 



Designer 


Design Engineer's 
Own Estimate 


Human Facto 


>rs Analysis 


Quantity 


Skill "Value " 


Quantity \ 


Skill "Value " 


Y 


4 


18 


6 i 


42 


L 


4 


12 


5 | 


39 


F 


4 


20 


7 ! 


47 


Mo 


4 


22 


6 ; 


42 


Mo 


5 


23 


7 ; 


49 


Ka 


2(inc) 


10(inc) 


5 i 


35 


K 


8 


52 


4 i 


26 


W 


3(inc) 


lT(inc) 


6 i 


44 


Ave rage 


Dl 


21.7 


. tJ. 
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7» A comparison of subject manpower estimates with those 
produced by more objective means (eg, Barton et al. , (1964) queuing 
model) is discussed in Appendix III* 

What can one conclude fzom this examination of manpower estimates? 
It cannot be said that the personnel restrictions imposed were out- 
standingly successful in forcing engineers to work within these limits. 
In some cases engineers felt the restrictions were unrealistically 
stringent and could be complied with only Ly forcing a very costly 
design. Others did not express this opinion. 

The authors would like to offer an hypothesis at this point. It 
is that engineers do not respond explicitly to personnel requirements 
(or, for that matter, to equipment requirements), but only in a 
general way. It will be recalled that six of the eight subjects 
reported that the personnel requirements did influence their design 
(and by extension their manpower estimates) . On the other hard, 
when questioned specifically abort the Influence of personnel 
requirements, they did not feel that these requirements played a 
major role in their design, pointing to reliability, system complexity 
and tum-around time requirements as the basic factors influencing 
their designs. The fact that a personnel requirement is imposed on 
engineers does not mean that they will necessarily design to the 
letter of those personnel limitations, but that they will take the 
personnel factor into greater account than they would if no personnel 
requirement had been imposed at all. This would account for their 
violation of manpower limitations in their estimates, and at the 
same time for the fart that subjects restricted to a lower skilf level 
had lower skill level estimates than those not r.o limited. From that 
standpoint the existence of a personnel constraint serves as a sort of 
benchmark to the engineer, who then tends to work within the general 
area of the mark even though he may deviate from it upon occasion. 

The same element may be present in the presentation of HRD inputs. 
RegartLless of the attention paid by engineers to specific HRD items, 
they still view the existence of the inputs as evid.ence that the 
procuring agency wishes them to pay more attention to personnel factors 
than they would otherwise. From that standpoint, too, the more HRD 
information supplied to the designer, the more conscious he will be 
of the need to consider personnel limitations in his design. 

Admittedly the fact that specific personnel requirements have 
only l general effect on design is somewhat frus .rating, but it 
represents rather accurately, more so than the vague generalities 
sometimes expressed in articles on the design process, the banner 
in which the engineer designs. Specific hardware requirements, lire 
reliability, do have a more measurable effeco (hence a seemingly 
greater effect), but even with such specific hardware requirements, 
the runge oi responses one finds (as Was discovered in the 1968 
study) indicates that hardware requirements are not completely 
effective either. 
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SECTION IV 



RECOMMENDATIONS 



It appears quite clear now, as a result of the present et.udy and 
the one preceding in (Keister, et al., 1968) that HKD inputs supplied 
to design engineers incrementally, following the issuance of the RFP/ 

SOW, will have little, if any, effect upon the design concept. The 
basic design concept is largely fixed within a short time after the 
issuance of the SOW. Although there is no guarantee that the engineer 
will use HRD inputs even when they axe presented under optimal condi- 
tions, it is quite clear that the present method of supplying them 
incrementally reduces the probability of their being used. 

Moreover, if the human resources specialist expects any consider- 
ation to be given by the engineer to his inputs, these inputs must 
be phrased as design constraints, they must be comprehensive and re- 
pressed in such a way that the engineer understands the design impli- 
cations of these inputs. 

The situation is by no means as bleak as the preceding paragraphs 
might suggest. Phrased properly and provided on e timely basis, 

HRD inputs do exercise some influence on design. The more design- 
relevant these inputs are, the more influence they have. 

The problem is one of being able to supply meaningful Human Resources 
design relationships. If appropriate personnel inputs (i.e*, mean- 
ingfully related to design) are supplied to the engi near, he will use 
them. The format of the inputs is less important then their content. 

At the very' least the RFP and the SOW must incline the following: 

(l) Description of the nannlng structure, for which the equip- 
ment is to be designed. Requirements must be specified for: 

(a) Maximum number of epei at ing/ioai nte nance personnel allow- 
ed to b i in the crew by job position. It should be clearly specified 
that any system configuration requiring personnel In addition to that 
number will be unsatisfactory. 

(b) Maximum skill level allowed for each job position. 

This skill level should be related to the specific tasks to be per- 
formed by personnel in the new system. 

(c) The function and task capabilities of these personnel. 
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(2) The design implications of the above must he clearly 
expressed* 

Although a few procurement documents do include information on 
the manning structure for new systems, none of them indicates the 
design implications* Indeed a major weakness of human resources 
support data (SD) is tha J few, if any, design deductions are drawn 
from the task, time-line and '^QPRI analyses which make up those 
data. 



Additional research is needed regarding a number of problems* 

(l) Methods are needed for determining early during the conceptual 
phase of a system the "likely" manpower force to be available luring 
the time period when the systec' would become operational. This 
"likely" force would be derived from sources such as phased out 
systems, career personnel, new enlistees and current training 
courses. (2) Techniques are needed for comparing the "likely" force 
with alternate forces of varying quantities and skill distributions 
and determining their impact on the cost, capability, reliability, 
availability, etc. of the system. Ultimately a "desired ,f manpower 
structure would be proposed for the system. (3) The design impli- 
cations of manpower requirements need to be fully developed, so 
that the design concepts and characteristics which will yield the 
"desired" manpower force can be specified in the contract stabemert 
of work. For this research to have maximum validity it should stud?/ 
the performance of personnel at operational sites in relation to the 
design concepts of the system. ( 4 ) Methods are needed for periodi- 
cally testing the design during conception and development 
compliance with the nampower requirement constraints, (5) Ir. addition 
to these improvements in Hki) methodology, a document is needed which 
could be used to supplement the SD analyses by specifying tin ir desie? 
impl icat ions . Such a document would then he used not only the 
human resources specialist to make design recommendations, hi t also !\ 
the Air Force manager and engineer to extrapolate the HRD inputr 
provided to specific design relationships. 
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APPENDIX I 



ABBREVIATED SCENARIO OF EQUIPMENT AND PERSONNEL 
INPUTS PROVIDED TO ENGINEER- SUBJECT 



NOTE TO THE READER 

The length of some of the equipment and personnel inputs provided 
to engineer -subjects in this study is so extensive that to have included 
all inputs in their entirety would have made this report extremely 
unwieldy. Consequently, less important inputs have been compressed by 
reproducing only that material which is Illustrative of the general 
character of the input. Inputs considered by the authors to be of 
major importance have been reproduced in their entirety. 

Where the purpose of a particular input or part of an input may 
have been unclear without additional explanation, explanatory material 
has been added in brackets. 



INTRODUCTORY SESSION 

Instructions for Participating Engineers 



The United States Air Force, through & contract with the Bunker- 
Rarao Corporation, is conducting a study to determine how engineers rake 
use of the information they are given (or develop themselves) to design 
a subsystem. Since any subsystem Is composed of two basic elements, 
equipment and people, we assume that the engineer has available to him 
two kinds of information: information about equipment requirements, 

characteristics, functions, etc; and information about or relevant to 
the personnel who will operate and maintain that equipment. 

The Air Force is interested in the engineer's use of both types of 
information, but it is particularly interested in the use made of 
personnel information. The reason is that although the engineer is 
p^ustomed by training and experience to using equipment information, 
personnel information may be relatively unfamiliar to him. The Air 
Force is interested in finding out if the personnel information it 
supplies to the engineer is used by him, and especial] v if that infor- 
mation makes a difference to the overall subsystem design. 

To answer these questions, it is necessary to present this infor- 
mation in the context of the development of subsystem. Short of 
actually conducting the study during the development of actual equipment, 



which would take an excessive length of time, the only other way of 
creating a developmental/design context is to reproduce or simulate the 
development of a subsystem in a highly abbreviated form* This simula- 
tion will naturally have to be of the paper and pencil variety. However, 
this does not concern us too much since we are interested in studying 
the very early design phases, before detailed drawings are made and 
equipment fabricated. 

What we have done is to take an already developed (operational) 
subsystem, extract the items of information used in its development and 
arrange them in a sequence which corresponds to the way in which they 
were actually used to design that subsystem. The subsystem selected by 
the Air Force is the ground maintenance subsystem for an air-to-ground 
missile carried by a B-52 bomber. The reason you were selected as 
subjects for this study is because you have helped to design similar 
ground maintenance subsystems. 

Obviously, such a maintenance subsystem is a very large one, and it 
would be impractical to ask you to try to design the entire subsystem. 
What we have done is to ask you to consider in your design only the 
electrical components of the ground maintenance equipment. Consequently, 
we have arbitrarily simplified the subsystem by ignoring certain equip- 
ments and operations which you, who are experienced in the design of 
such subsystems, will obviously note. Do not be disturbed by this. The 
subsystem is supposed only to represent maintenance subsystems in 
general. 

At the close of ihis introductory session you will be given design 
statement of work which contains certain equi;ment and personnel re- 
quirements and information. On the basis of these design requirement:, 
plus additional information which you will receive at the start of each 
subsequent session, you will design a ground maintenance subsystem 
which best meets the requirements in jour statement cf work. 

Since you are performing a conceptual design study, ve ask you to 
consider three possible design concepts: one appropriate to a manual 

system, one for a s emi-automatic system, one for an automatic system. 
Although any definition of these terms can only be loose at best, ve 
define a manual system as one requiring a rather extensive involvement 
of personnel in the system operation. An automatic system is one which 
requires relitively little personnel involvement, and a semi-automatic 
system involves equipment and personnel functioning in about equal 
amounts • 

We ask you to consider th design requirements in terms of all 
three system configurations and to describe the system you would design 
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if you decided to go automatic , semi -automatic or manual. We will also 
ask you to trade off the various equipment and personnel factors involved 
in each configuration and tell us why you selected a particular con- 
figuration as being the best* You will also tell us which items of 
information you found most useful and why. 

Since we will have only k working sessions, and since designing a 
complete subsystem is a big job, we will ask you to go into only enough 
detail to indicate the general character of the equipment you would 
design or purchase* We particularly went to know such things as: 

(1) How many and what types of equipment (both special purpose and 
off-the-shelf) will be used by system personnel; 

(2) The outstanding characteristics of that equipment and how 
they are intended to function; 

{3; How the equipment will be used by personnel; 

(k) Hov many men and of what type will be needed to use the 
system* 

You will not be asked to develop detailed equipment drawings . How- 
ever, you should sketch any equipment to be designed in enough detail 
to let us know what you have in mind* 

One thing I should emphasize. The questions we ask and the tasks 
we ask you to perform are not tests in the conventional sense of the 
work* Ihe word "test” uggests that only one correct response can be 
trade to these design problems. In these design problems there are no 
correct or incorrect answers; because only you can tell us what the 
correct answer should be. For this reason it is most important that, 
although we cannot completely provide all the conditions under which 
you ordinarily design, you respond to these problems in the way in 
which you would ordinarily solve an actual design problem* Remember 
that the value of the inf conation you provide depends cn how accurately 
it reflects the way you ordinarily design on the job* Remember also 
that this is not a test of your ability, although we want you to do 
your best. We would not have selected you to do this work if we did 
not think yo\i could do it. 

We will probably meet once a week and the schedule will be adapted 
to your convenience. Between our sessions you may, if you wish, refer 
to the inputs you have been given. However, this part of the study is 
purely voluntary* During your sessions and in the interim, you may 
consult anyone inplant from whom you wish at lltlonal information. We 
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do ask one thing of you, however; do not confer with your fellow parti- 
cipants in the study on any aspect of the study* To do so would 
seriously reduce the value of the results* 

Are there any questions? 

Here ifi the Statement cf Work which you as the project engineer for 
the AGM-X ground maintenance system will have to work to. We would like 
you to take it with you and to examlnfe it carefully. Please bring it 
with you when you return for the first session. 
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STATEMENT OF WORK 



AEROSPACE GROUND EQUIPMENT (ELECTRONICS) 
FOR MAINTENANCE OF THE AGM-X 



1.0 PURPOSE AND SCOPE 

1 .1 P urpos e 

This statement of work (SOW) establishes the requirements for the 
conceptual design of the ground maintenance equipment for the AGM-X 
system, including any peculiar checkout, maintenance and test equipment 
required. The AGM-X system is a short range attack missile which con- 
sists of the AGM-X air to surface missiles including nuclear warheads, 
B-52G and H carrier aircraft avionics equipment (CAE) , carriage/launching 
mechanisms, support and training equipment, facilities, data and 
personnel. 



1.2 System D esc ription 

1.2.1 Missio n 

The operational mission for the AGM-X carrier/missile weapon 
system is oriented toward the strategic objectives of the nation's 
general nuclear war forces. 

The details concerning the concept of weapon system deployment, 
system performance and capabilities, and program scheduling, are 
classified. 

1.2.2 Carr i er Aircra ft 

The B-52G&H aircraft can carry eight missiles internally on a 
rotary launcher with four MK-28 bombs in the bomb bay, and 12 missiles 
externally (six per pylon). Carrier avionics jncude the bombing 
navigation equipment, an inertial measurement unit, a master computer, 
the radiating site target acquisition system (RASTAS), and controls 
and displays necessary for the operation, control and launch of the 
AGM-X missiles. (Location and descriptions of these equipments are 
provided in the Appendix.) 

1.3 Scope 

Ihe contractor shall conduct feasibility or trade studies for the 
design of an electronics checkout subsystem required to perform a 
complete operational check of missile and CAE electrical systems, 
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including the maintenance functions of malfunction isolation, calibra- 
tion and checkout of malfunctioning AGM-X missile system components and 
subassemblies. The MGE requirements presented herein are limited to 
electrical/electronic equipment necessary for indirect field site sup- 
port of the various system checkout and launch functions performed by 
OGE. 



1.3 Scope 



The contractor shall conduct feasibility or trade studies for the 
design of an electronics checkout subsystem required to perforin a 
complete operational check of missiles and related CAE systems, in- 
cluding the maintenance functions of test, malfunction isolation, cali- 
bration and checkout of malfunctioning AGM-X missile system components 
and subassemblies. Based upon these feasibility or trade studies, the 
contractor shall select, describe, and design the optimum system for 
satisfaction of system design goals. 

The Maintenance Ground Equipment (MGE) requirements presented here- 
in are limited to eleotrical/electronic equipment necessary for indirect 
field site support of the various system checkout functions. 

1 , 3 • 1 Assumptions and Prerequisites 

1. 3*1*1 For AGM-X, the conventional three levels of maintenance will be 
used and will be compatible with existing maintenance proce- 
dures and facilities* 

Organizational naintenance down to the lowest replaceable unit 
will ue performed by teams and specialists on the flight line 
and in the hangar* flight line functions include removal and 
replacement of carrier aircraft equipment, installation and 
removal of single missiles or launcher/missile packages, inte- 
grated checkout of carrier and missiles and installation/ 
replacement f exchange of warhead. Organizational level lunr'ions 
in the hangar include missile checkout, removal and replace- 
ment of missile components, missile installation and removal 
from launcher, and installation/ replacement of warhead. 

Missiles will be repaired by removal and replacement of 
modular units after testing with a fault locater. Missiles 
are removed from the operational cycle for test and checkout 
only after onboard carrier tests show a malfunction* Periodic 
maintenance consists of a pre-flight and post-flight inspection 
on the carrier and replacement of time change items such as 
missile battery and cartridge-activated devices* 
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1. 3*1*3 Maintenance functions to be performed at the field sate repair 
shop will include malfunction isolation, calibration and check- 
out required for the repair of faulted subassemblies vhich have 
been removed from airborne and ground systems for repair and 
calibration. 

1, 3*1*4 (Eliminated) 

1.3 *1*5 MGE shall be available for malfunction isolation, calibration 
and checkout in the field site repair shop* 

1*3*1 *6 Where required, calibration capabilities sAall be specified 
for instruments and MGE, * 

* 

1. 3*1*7 Items of MGE shall be of such variety and Sufficient quantity 
to perform the maintenance functions required tc fault isolate 
and restore to operating condition, AGE and airborne items of 
electrical/elec tronic equipment vhich are designated field 
repairable. ' 

1.3 *1.8 Electrical/electronic MGE is required to provide a means for 
functionally verifying the signals from the' missile/pylon/ 
launcher interface to the CAE subsystems. She following 
capabilities must be included in the MGE,: ; 

(a) Verify the programmed events in a predete mined chrono- 
logical, timed sequence end supply the resulting signals 1 
to the umbilical interface. 

> ,i 

(b) Verify the s+atus of prerequisites to each event. 4 

(c) Respond in a predetermined, manner to 'hold and recycle ,• 

functions. ~ J 

(d) Verify correctly programmed stimulus signals and eval- 
uation of vehicle responses during vehicle checkout. J 

(e) Verify correct evaluation of vehicle discrete and analog 
functions which are used to determine vehicle readiness f 
status. 



1.3. 1*8.1 In addition to the MGE required to accomplish the above 

functions , a simulation device will be required vhich will i 
allow CAE equipment to be functionally checked without 
missiles/launchers/pylons being present. This simulator 
shall have the capability of el ctrically ; simulating missil ?s, 
launcher, and/or pylon functions , ! 
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1 . 3 Re quirements 



'Hie contractor shall ensure that at a minimum the following aspects 
are considered in these trade studies: 

( 1) Cost (Hardware costs only) 

(2) Equipment maintainability (see paragraph 8.0) 

(3) Equipment reliability (see paragraph 7*0) 

(4) Number and type of personnel 2 'equired (see paragraph 5.0) 

( 5 ) Performance efficiency 

(6) Safety 

The trade studies will analyze three alternative equipment configur- 
ations: manual, semi -automated, and automated, and document the reasons 

for selection of the contractor^ chosen alternative. 

2 • 0 Appl i cable Document s 

General - Ihe following documents form a part of this specification 
to the extent specified herein. In the event of conflict between the 
requirements of this specification and any document referenced herein, 
the requirements of this specification shall govern. References in the 
following documents will be considered only as a guide. 
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Technical Order? 



Manuals 



Specification: 
MilJ tary 



o 




T. 0. 00-20K( series) - Inspection and AGE Control, 

USAF Equiprrent 



AFSCM 80-3 



AFSCM &0-6 



AFSCM O0-8 



AFSCM 80-9 



AFSCM 127-1 



- Handboc V of Instructions 
for Aerospf.ee Personnel 
Subsystem Designers 

- Handbook of Instructions 
for Aerospace Ground 
Equipment Designers 

- Handbook of instruction 
for Missile and Space 
Vehicle Design, Vol. 1 
Ballistic Missiles 

- Handbook oj 1 Instructions 
for Aerospace Systems 
Design, Vol. 1 General 
Design Criieria 

- System Safety Management 



AFSCM 375-5 



- Systems Engineering 
Management Procedures 



MIL-T-152B Treatment, M isture - and 

Fungus-Besr.stant , of Com- 
munications, Electronic, 
and Associated Electrical 
Equipment 

MIL-S-351SB Support Equipment, Aero- 

nautical, Special, General 
Specif icat ion for the 
Design of 
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MIL-S-38130 

Standards 

Federal 

FED-STD-595 
1 Feb. 1961 

Military 

MIL-STD-143 

MIL-STD-454A 

MIL-STD-808 

MIL-S r fD-210M 
MI L“S TD-l47 2 

MIL- STD- 4 r f0 



Safety Engineering of 
Systems and Associated 
Subsystems, and Equipment, 
General Requirements for 



Colors 



Specifications and Stand- 
ards, Order of Precedence 
for the Selection of 

Standard General Require- 
ments for Electronic 
Equipment 

iiniches, Protectives, 
and Codes, for Finishing 
Schemes for Ground and 
Ground Support Equipnent 

Climatic Extremes for 
Military Equipment 

Human Engineering Design 
Criteria for Military 
Systems, Equipment and 
Facilities 

Maintainability Program 
Requirements (for systems 
and equipments) 
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3 # 0 Engineering Inspections 
3*1 Preliminary Design Review 

The contractor shall conduct a preliminary design review not ]ater 
than 60 days subsequent to award of the contract* This review shall be 
in accordance with AFSCM 375~5 j and shall be subject to approval of the 
AGM-X Project Office* 

3 • 2 Critical Design Review 

The contractor shall conduct a critical design review 180 days after 
award of contract. This review shall be in accordance with AECCM 375-5 > 
and shall be subject to appro val of the AGM-X Project Office. 

3 i 3 F inal Acceptance 

Final acceptance of the contractor’s work shall be indicated by 
accomplishment of a DD Form 250 reflecting the technical acceptance of 
the designs provided by the contractor and completion cf all contractual 
requirements as specified in this SCW and associated documents* In the 
event there are exceptions to the acceptance reflected on the DD Form 
250 or attachments thereto, the contractor shall be required to correct 
all exceptions as specified within the time limit mutually agreed upon 
during the execution of the DD Form 250. 

4 .0 Performance Requirements 

4.1 Checkout and Test Equipment 

Support of the missile and associated carrier ai rcraft system shall 
be accomplished through the use of checkout equipment which shall enable 
the operator to perform a gross operational performance analysis o? the 
system. Test equipment shall be identified as that required to perform 
a more detailed analysis of the system down to the lowest level of re- 
placement unit. 

4.2 Checkou t and Test Time 



The time required to connect the checkout and teat equipment to the 
missile and/or associated missile-carrier aircraft system, warmup, 
conduct the required tests, and disconnect the equip u._nrG shall be held 
to a minimum, and in no case (other then malfunction correction) shall 
exceed two hours. Lightvei^it cable assemblies and connectors, and 
quick disconnects, shall be used in order to facilitate ease cf handling, 
connection and disconnection. 



4,3 Degree of Testing 

Die test method utilized shall employ the minimum number of tests 
necessary to check out the missile and associated missile-carrier air- 
craft system or isolate a malfunction to the lowest replaceable unit 
(LRU). 



4 • 4 Test Tol erances 

The extent of test tolerances shall be limited to that necessary to 
establish realistic acceptance or rejection criteria for the missile and 
associated missile-carrier aircraft systems based on operational re- 
quirements. The major test tolerances shall further be predicated on 
the operational tolerances of the systems. The test equipment shall not 
be required, to test to the design tolerances of the system, except in 
instances where design tolerances and operational tolerances are identi- 
cal and/or can be obtained without additional penalty. 

4 • 5 Communication 

Equipment shall be provided for communication between personnel. 

4.6 Int erc onnections and Cable s 

The contractor shall give consideration to the utilization of the 
interconnecting devices which are compatible with the missile, carrier- 
aircraft, and associated maintenance equipment. This will include 
electrical, hydraulic, and penumatic interconnections, as well as hitches 
towbars and full servicing vehicle fittings. Interconnecting cables 
shall be provided as necessary to connect the test equipment to the 
nissi^e/aiicraft system, and to any portable antennas, etc., that are 
requir,d for flight line maintenance. Cable lengths shall be sufficient 
to permit positioning -the checkout equipment so as not to interface with 
normal- missile/aircraft servicing during checkout of the system. Pro- 
visions shf 11 be made for storing all interconnecting cables and other 
accessories within the checkout equipment. 

4 . ' i AGE F. i ze and Weight 

The contractor shall give consideration to the design of AGE with 
regard to weight and size (reference MIL STD-1472) . Checkout and test 
equipment should be transportable by the minimum number of personnel and 
should be capable of being carried aboard the aircraft through existing 
access docu:s. 



4 . 8 AGE Maintenanc e 



AGE shall bs designed, wherever possible, to utilize non- repairable 
(throw-away) components to facilitate ease of maintenance and to minimize 
the requirement for detailed maintenance logistic considerations. The 
contractor shall give consideration to the maximum utilization of the 
modular concept in the design of AGE to facilitate fault isolation and 
maintenance repairs within the AGE. 

4.9 Calibration 



Electronic and electromechanical AGE shall be designed to permit 
calibration at specified intervals. Test points will be provided to 
permit calibration. 

4.10 Fl ight Line Che c kout and Test Equipment 

The flight line checkout and test equipment shall provide the opera- 
tor with the capability to analyze the functional performance of the 
system, subsystems, and components statically and/or dynamically with 
the optimum degree of accuracy in test results. Utilization of standard 
and commercial equipment is encouraged. Checkout and test equipment 
shall perform integrated system tests where integrated airborne systems 
are utilized. Whenever a dynamic testing concept is employed, the AGE 
shall be capable of presenting test problems representative of those 
encountered by the systems in operation in order to determine the system 
performance level under normal environment. Testing, whether static or 
dynamic shall be sufficiently comprehensive to analyze and isolate to 
the LRU level, in the optimum period of time, the functional performance 
of the airborne systems. Testing and monitoring displays shall be so 
designed as to minimize inteipretation by operators. 

4.H Electronic Electrical AGE 



Flight line electrical/electronic AGE will be designed to perform 
the following: 

(1) Verification ol hazardous current safety when electrically 
mating the missile: or launcher/missiles to the carrier- 
installed system. 

(2) Insertion and verification of mission data into the carrier 
master computer prior to committing an aircraft to alert. 

(o) Simulation of nissile or missiles/launcher to penult integrated 
test of carrier: C,\E prior to actual installation of the nissile 
or missiles/lanncher. 
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(4) Verification of system integrity. 

(5) Fault isolation to the Line Replaceaoie Unit le\el. 

4 • 12 Shop Checkout and Test Equipment 

The primary utilization oi shop checkout and test equipment shall 
he that required for the more detailed analysis of systems, subsystems, 
and/or components that have been removed from the missile/carrier-- 
aircraft after being diagnosed as malfunctioning and placed in the 
environment of the field shop. This type of AGE will enable the tech- 
nician to test the airborne equipment to the lowest level of removable 
units with the maximum practical accuracy* Periodic maintenance and 
system overhaul shall also be performed with shop checkout/test equipment. 
This equipment shall provide the following: 

(a) Performance of routine maintenance tests and complete 
functional tests of line rejected units. 

(b) Detection and location of malfunctioning module assemblies 
of line reject 3d units. 

(c) Facilities for the replacement, adjustment, calibration, and 
repair of defective line rejected units. 

(d) k self-checking capability for use in checking the shop check- 
out and test equipment without the use of any ancillary test 
equipment. 

4.13 Electrical Equipment 

All electrical equipment shall confo' m to the general requirements 
of AFSCM 8 0-6 and specification MIL-S-85I2 and shall be compatible with 
the aircraft and missile systems. Electrical power equipment designed 
for 400 cycles, 3 Phase, 208 VDC shall be compatible with the power 
characteristics defined in specification MIL-3TD-7^>. Electrical systems 
designed fo ' operations cn 220V, 3 Phase, 60 Cycles, snail be readily 
convertible w > 440 V, 3 Phase, 60 Cycles, without replacing compose. lets. 





77 



5.0 



Personnel Subsystem 



5.1 Personnel Manning Requirements (for C-N Group) 

Use of AGE for the maintenance and testing of any individual 
system, subsystem or component of the missile and associated missile- 
carrier aircraft ^hall not require more than tvo operators- -one at or 
in the aircraft and one at the checkout or test equipment. 

5.1 Personnel Manning Requirements (for 0-S Group) 

Equipment shall be designed to be operated by military technic iaxis 
with an Mr Force Speciality Code three-level skill only (see Appendix 
for definition of skill levels). 

5.1 Personnel Manning Requirements (for Incremental Group) 

Equipment Design shall minimize the quantity and skill level of 
military personnel required to operate the equipment. 

5*2 H uman Engineering 

As outlined in MIL-STD-IV 7 2 , the contractor will apply human, 
engineering to hardware and system design to assure optimum operation 
and maintenance, utilization of the human as a component in 'the system, 
and reduction of tasks affected by human limitations to a minimum. This 
will include human design considerations for maintenance, operations, 
comii. unicat ions , illumination, noise level, reliability, safety, climate 
and environment (Ref. MIL-STD-1472) , Studies and recommendations will 
be directed by AGM-X Project Office for the improvement of procedures 
and design as inefficient operations situations are detected. 

6.0 Safety 

All designs shall incorporate maximum protection for operating arid 
maintenance personnel against hazardous conditions. Adequate provisions 
sha. 1 ! be made to vam and/or protect personnel and equipment against 
injury and damage. All designs shall be reviewed by qualified engineers. 

7.0 Reliability 

Ihe AGE subsystem shall have a minimum me an- time -between- failure 
(MTBF) of 400 hours when operated under rhe environmental conditions 
specified in Table I. Failure is defined as the inability of the AGE 
subsystem to perform within the limits specified. 
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8.0 Ma intainability 

Tile contractor shall establish a maintainability program in accord- 
ance with applicable sections of ML- STD- 470 Maintainability Program 
Requirements. The terms and definitions for maintainability not other- 
wise described or delineated shall be in accordance with MIL-STD-778* 

As a design goal, the M1E of AGM-X shall incoiporate factors that 
enhance its ease of maintenance and accessability. The maintainability 
Characteristics equipment, inspection, servicing, test, replacement and 
overhaul operations required to restore operational capability with a 
minimum expenditure of time, men and materials. When necessary to 
accomplish this requirement, special tools and service equipment shall 
be identified* The inclusion of maintainability characteristics as an 
inherent feature shall occur simultaneously with initial design and shall 
be continually analyzed and controlled throughout the development cycle. 
The equipment shall be designed so chat the following system mean main- 
tenance time shall not be exceeded: 

Mean Corrective Maintenance Time (M^) > 1*0 hour within 3*0 hours 

= 1 <T. 
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TAHLE XVII 



ENVIRONMENTAL CONDITIONS 



Environmental Storage, Transportation, and 



Conditions 


Operation 


Handling Limits 


Low temperature 


-65° minimum 


-80°F (maximum duration 24 
hours) followed and preceeded 
by equilibrium at - 40 °F 


High temperature 


125°E with a 
daily maximum 
of l 60 °F for 
4 hours 


l60°P for 4 hours daily 


Temperature Shock 


Not applicable 


- 80 °F 1 0 + 125 °F within 5 
minutes 


Humidity 


5$ to 100$ 
relative 
humidity with 
condensation 
at 85°F and 
below 


Same as operation 


Atmospheric 

pressure 


15.4 psia to 
7.1 psia 


15.4 psia to 1.68 psia with 
maximum rate of descent of 



5C00 feet/min« and maximum 
rate of ascent of 2500 feet/ 
min, 



FURTHER REQUIREMENTS FOLLCW 
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The following material was appended to the statement of work, It 
was available to the Omnibus group as part of the basic SCW, and vas 
provided as a separate input to the Incremental group in Session 3« 



APPENDIX 



The material provided in this Appendix contains information secured 
from design analyses performed by the Air Force. It presents information 
which the government feels will be useful in performing previously 
specified contractual activities. Ihe Appendix is, however, to be viewed 
as being advisory only, and should not be considered as a contractual 
requirement. 
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DESCRIPTION OF THE AGM-X MAINTENANCE MAN 
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The System Specification establishes that airmen with certain basic 
skills and a specific level of proficiency must be able to operate end 
maintain the AGM-X weapon system, Tuese airmen constitute a very critical 
elen .it of the system. An understanding of this element of the Personnel 
Subsystem is essential to design of the AGM-X Weapon System, 

To assist in defining what kind of individuals will be working with 
the weapon system, the following discussion describes the skill level 
codes used by the Air Force, Some historical a ■‘formation is also 
included. 

Air Force career fields are identified by t. first two number;; in 
the airmans Air Force Specially Code (AESC) , such as I+3XXX indicates 
Aircraft Mechanic, 46XXX indicates Weapons Mechanic* All officer spe- 
cialties are explained in AFM 36-1 and all enlistee specialties are 
explained in AIM 39“1* 

Possibly, the most significant position in the AK'C designation 
number is the fourth position, e,e,, 46X3* or 31X£X# The fourth nuiber 
establishes the skill level of the individual described by the AFSC 
This skill designator is the key to designing a weapon system which. can 
be maintained by the personnel spelled out* The skills assigned by the 
specification and described in AIM 39-1 are very broad in scope. Seldom 
do individuals qualify in all the required ai*eas. Pre.ssures of war, 
enlistment rate, etc*, more frequently than riot will cause downgrading 
of technical qualifications. j 



AIR F ORCE MAN NI NG - AUTHOR I ZED VS ACH JAL 

The authorized numbej's, end the skill level of the personnel to be 
assigned to a given AF unit, are indicated numerically as 3 , 5* 7 ad 9 
leve] and cen be translated as follows; 1 

3 level - Kelper/App rent ice 

5 level - Sp>ecialist/Mechanic j 

7 level - Technic ian/Supervis or . 

9 level - Superintendent 

i 

A further translation is from indicated skill level to actual j 
technical experience and Is approximated as follows: j 

3 level - A basic AF technical school plus one year maintenance 
experience. 
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5 level - A basic AF technical school plus training on a specific 

weapons system plus one to five years overall experience, 

7 level - Additional training and five or more years experience, 

9 level - Additional training and 10 or more years experience. 

After graduation fro" a basic technical course, the three-level 
airman will generally be assigned to work location at some operational 
base. His job assignments will be primarily removing and replacing 
components using technical manual procedures. The three-level airman 
is not permitted to deviate from procedures provided by technical manuals. 

An airman with five-level skills has progressed through the three- 
level skill in the same or associated career field. He will have com- 
pleted two to three years of work experience at the three-level and 
received some additional technical training and or.- the- job training. 

Most airmen holding a five-level skill have attended one additional 
technical school which was oriented to a specific weapons system. The 
training cour«<: was probably one to two months in length for electro/ 
mechanical fields and two to five months for electronics; and provided 
specific information on operation, trouble analysis, checkout and repair 
of equipment for whcih he will be responsible, Kxperience in the five- 
level skill consists of trouble analysis, repair and checkout using 
voltmeters, electronic scopes, and tape programmed checkout equipment to 
take specific measurements. All activities of maintenance are directed 
by technical manuals, but, in some cases, may require use of basic 
theory for interpretation and analysis. 

The five-level airman has been in the service three years, of which 
approximately 10 months were taken up in training. This leaves a 26- 
month period of time in which he could be considered actively engaged in 
his "trade," However, because of non-technical military activities and 
Air Force work load scheduling problems, the average direct labor man- 
power utilization rate is approximately 45$. Khen this 45$ is factored 
into the 26 months not in training, the total experience gained in both 
the three-level and five-level skills is U .7 months. In 84.6$ of the 
cases, the airman is not highly motivated toward a military career \nd 
will take his discharge in one year. 

An airman carrying seven-level skills is very likely to be a 
career airman. He has been in the service five years or more and holds 
the rank of S/Sgt. or T/Sgt. 
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PERSONNEL AVAILABILITY 



The personnel to be available at the beginning of AGK-X '•raining 
are anticipated to be from B-52 and B-58 units, 'lie following personnel 
Eind related experience levels are anticipated: 

Missile Electronic Maintenance Technicia n 

3165O/XX 1 year experience on a similar or related missile system 

Munitions and Weapons Maintenance Technician 

46250 /XX 1 year experience on a similar or related missile system 
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PRELIMINARY QUANTITATIVE AND QUALITATIVE 
PERSONNEL REQUIREMENTS INFORMATION (QQPRl) 



Analysis of the functional requirements of ground maintenance of the 
AGM-X system has resulted in the following preliminary descriptions of 
the personnel needed to perform ground maintenance* 

The following Air Force Speciality Codes (AFSC) are required for 
performance of the integrated system checkout of the missile and carrier 
aircraft equipment. 

The integrated checkout team will consist of the following AFSC's: 
1S15B, 31650 /YX, 46250/xx. 



1, 1515B. Operations Support Officer 

The Operations Support Officer assigned to the Plans branch of the 
Wing Operations and Plans Division is responsible for developing the 
AGM-X mission plans from source data provided by higher headquarters. He 
is responsible for the loading of the mission tape data into the carrier 
aircraft master computers. He supervises the Missile System Analyst 
Specialist/Technician, AFSC 31650/70, in the operation of the data 
inserter monitor set (DIMS) during Mission data loading. 

2. 3l650/XX. M issile Systems Analyst Specialist/Technician 

The Missile Systems Analyst Specialist/Technician is responsible for 
AGM-X maintenance activities at both the flight line and integrated main- 
tenance facility. 

At the flight line, he performs data loading into the carrier master 
computer using tho operational, mission, and training tapes and the AGM-X 
data inserter monitor set. He performs integrated AVE/CAE tests on the 
AGM-X missile carrier aircraft systems. He performs scheduled inspections 
and organizational level maintenance of AGM-X missile launch control 
system and AGM-X carrier aircraft equipment. 

In the integrated maintenance facility he prepares for and performs 
missile checkout and isolates missile malfunctions. He disassembles 
missile sections, removes and replaces faulty electronic components, and 
verifies repair. Ke tests, removes, replaces, and repairs faulty missile 
wiring. He removes and replaces the electronic and flight control 
sections during rocket motor replacement. 

Ihe Missile Systems Analyst Spec ialist/Technician performs bench 
testing of the AGM-X missile and carrier aircraft system components. He 
removes and replaces assemblies or subassemblies in the components and 
verifies component repair. 
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3. AFSC 4 62 50/ XX. Weapons Mechanic /Maintenance Supe r visor 

As a member of the AGM-X Weapons Loading Team, the Weapons Mechanic/ 
Maintenance Supervisor performs missiie-to-carrier hazardous current 
checks and uploads and downloads the AGM-X Dissile, the launcher/missile 
package, the pylon/missile package, and bomb rack, in the B-52 aircraft. 

He perfoms/verifies ordnance and warhead safing and enabling during 
ground operations. He performs payload to missile hazardous current 
checks and in stalls/ removes payload sections at the flight line. He 
observes and verifies launcher rotation during integiated system test. 

He transports the missiles, pylons, launchers, payload sections, and bomb 
racks to and from the flight line, integrated maintenance facility, mis- 
sile/munitions storage facility or nuclear weapons cturage facility, and 
empty pylons from the CAE/aircraft preflight maintenance facility to the 
integrated maintenance fecility< 

He performs organizational level maintenance on the launcher, pylons, 
bomb rack, AGM-X weapons status and control panel, anc. AGM-X consent panel 
in the aircraft. He performs organizational and field level maintenance 
on the munitions handling and transportation AGE. 



SKILL LEVEL DEFINITIONS 



The following is a definition of the Air Force skill levels referenced 
in this istatement of work: 

3 level --Usually acts as helper or assistant ^ but can do simple tasks on 
his own, such as simple checks. Performs simple manual operations readily 
(without assistance) but requires assistance (supervision or use of 
manuals) with more complex operations* The following activities are 
characteristic of the 3 level maintenance technician: 

(1) Performs simple preventative maintenance without supervision; 

(2) Performs emergency responses only when advised to do so by 
higher level technician; 

( 3 ) Removes and replaces modules under supervision; 

(4) Perform uotentially hazardous checks under supervision; 

( 5 ) Monitors and records equipment status values from displays; 

(6) Performs programmed equipment checks in accordance with written 
procedures; 

(7) Makes simple (discrete) electrical connections without 
supervision* 

5 level --Performs most maintenance activities with the help of the 
3-level* He tray require assistance (supervision or use of a checklist) 
with more complex operations, particularly those requiring significant 
decisions or a high degree of hazard. The following is characteristic of 
the 5-Level maintenance technician: 

(l) Is capable of performing all activities of the 3 level techni- 
cian; 

(а) Performs potentially hazardous checks with minimal supervision; 

( 3 ) Removes and replaces modules without supervision; 

(4) Decides what equipment checks should be made and when they 
should be taken; 

(.5) If test equipment malfunctions, corrects malfunction vnen crew 
chief not available; 

(б) Assists crew chief in performance of complete subsystem check- 
out; 
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( 7 ) Capable of performing most troubleshooting activities; 

(8) Analyzes malfunction displays to diagnose equipment failures; 



(9) Coordinates info mat ion from multiple displays to assess sub- 
system status; 

(10) Supervises 3 level technician vhen crew chief not available* 



7 level --Performs all tacks including those involving significant 
decisions and hazardous operations* The fokLowing activities are charac- 
teristic of the 7 level maintenance technician. 



(l) Is capable of performing all activities of lover level 
technicians; 

(p) Supervises lower level technicians and is responsible for all 
crev activities; 

(3) Perfom s highly hazardous checks on own responsibility; 

(4) Makes special purpose, elaborate electrical hookups; 

( 5 ) Takes responsibility for performance of complete subsystem 
checkout; 

(6) Capable of performing all troubleshooting activities. 
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Figure 110. Time Line Analysis: Shop Maintenance 
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REQUIRE MENTS ALLOCATION SHEET 



FUNCTION 
N AME 



DESIGN REQUIREMENTS 



PREPARE 
CARRIER FOR 
MISSILE(S) 
LOADING 



1 . 



2 » 



TEST LINE 3. 
ITEM 1 



PERFORM 
ORGANIZA- 
TIONAL main- 
tenance 



4. 



5, 



6 . 

7. 

8 * 



(U) Prior to loading missilefs) and/oz launchers onto the carrier, 
it is required that a hazardous current check be made of the CAE. 

(U) An adapter is required to provide an electro-mechanical inter- 
face between the hazardous current checker and the side of the 
missile, Necessary switching is required in that adapter to per- 
mit the hazardous current checks. 

(U) An integrated AVE-CAE te^t will be run on the carrier- 
installed systems prior to committing it to ready alert status. 

This same integrated test will al*»o be run on carrier - installed 
CAE with a simulator replacing the launcher /missiles assembly. 
This test, in conjunction with certain additional fault isolation pro- 
cedures, will establish the GO - NO GO status of each missile, 
the rotary launcher, and line- replaceable CAE. 

a, (U) Test equipment shall be identified as that required to 
fault isolate the system to the lowest level of base replace- 
able unit. 

b. (U) The test method used shall employ the minimum number 
of tests necessary to checkout the missile and associated 
missile- carrier aircraft system or locate a defective re- 
placeable unit. 

a, (U) The time required to connect the checkout and teat 
equipment to the AGM-X System or Subsystem, to warmup, 
to conduct the required tests, and to disconnect the AGE 
shall be held to a minimum. This time shall be compatible 
with the specified aircraft turn-around time. 

b. (U) To facilitate ease of handling, connection, and discon- 
nection of the AGE, light-weight cable assemblies and con- 
nectors shall be used. 

(U) The test tolerance within which the AGE checks and tests the 
missile and associated missile-carrier aircraft systems, shall 
be based on the operational tolerances of the system. 

(U) Equipment shall be provided for verbal communication be- 
tween operators, 

(U) Transportable checkout r.ndte6t equipment 6hall be of a size 
and weight to allow handling by as few men as possible and to 
allow carrying aboard the aircraft through the normal access 
doors. 
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REQUIREMENTS ALLOCATION SHEET 



FUNCTION 

NAME 

PERFORM 9. 
ORGANIZA- 
TIONAL MAIN- 
TENANCE 

10 . 



11 . 



12 . 

13. 



CHECKOUT 14. 
AGM-X CAE 

i5, 



16. 




DESIGN REQUIREMENTS 

(U) AGE shall make maximum use of non-repairable (throw away) 
modules and components to facilitate ease of maintenance and to 
minimize the requirement for detailed maintenance logistic consid- 
erations. 

(U) All electronic and electro -mechanical and e lect ro - pneumatic 
AGE shall be subjected to a calibration process at such intervals as 
will be commensurate with the functional requirement of the equip- 
ment. 

a. (U) All electrical equipmert shall conform to the general re- 
quirements of AFSCM £0-6 and Specification MIL-S-3512 and 
shall be compatible with the aircraft and missile systems. 

b. (U) Electrical power equipment designed for 400 cycles and 

28 VDC shall be compatible with the power characteristics de- 
fined in Specification MIL-STD-704. 

c. (U) Electrical systems designed for operations on 220V, 3 
Phase, 60 cycles shall be readily convertible to 440V, 3 Phase, 
60 cycles without replacing components. 

(U) The mean corrective maintenance time for the AGM-X missile 
shall not exceed minutes. 

(U) A positive means shall be provided to assure that the missile 
is in a '‘safe" condition prior to maintenance operations such as 
missile or missile rack unloading, missile or missile rack trans- 
porting, or missile checkout. The missile shall remain in a 
''safe 11 condition during these operations. Safe - All missile ord- 
nance devices shall be mechanically and electrically in safed con- 
dition. 

(U) A requirement exists to have the capability to perform an 
AGM-X CAE checkout without missiles aboard. 

(U) The CAE checkout shall: 

a. Verify a suspected malfunction, and 

b, Verify a replacement or repair of CAE installed in the carrier. 
(U) The CAE checkout shall verify *he following components: 

a. Master Computer, 

b. Inertial Measurement Unit, 

c. Display and Control Panels, 

d. Processor & Distribvtion Unit, 
c. ECU, 

f. Cabling 
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REQUIREMENTS ALLOCATION SHEE T 



FUNCTION 

NAME DESIGN REQUIREMENTS 



17. (U) A means is required to electrically simulate a missile when no 

missiles are aboard the carrier. A means is required to electric- 
ally eimulate the launcher /mis sile package when no launcher /mi s - 
sile package is installed on the carrier. The simulator must be 
functionally capable of: 

a. Receiving input data and supplying simulated output data. 

b. Receiving missile alignment commands and providing simu- 
lated alignment status. 

c. Providing simulated missile status information for verifica- 
tion at the carrier displays and carrier computer. 

d. Accepting and returning SAF discretes to the carrier. 

e. Monitoring carrier power and power control signals. 

f. Accepting missile release mechanism commands and return 
appropriate status responses. 

g. Electrically simulating rotary rack response to positioning 
signals. 
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SESSION 1 



Instructions to Participating Engineers 

Now that you've had a chance to go over the SOW, we want you to de- 
velop the basic design concept of a system which will meet the require- 
ments specified in the SOW. Among these requirements, ycu will note, 
are those which refer to the number and type of personnel. We ask you, 
therefore, in developing your design to particularly keep in mind the re- 
quirement to design to minimize personnel needs . 

In describing this design concept, you should include the following: 

(1) The number and types of major equipment items you would need; 

(2) What each equipment would test; 

(3) The characteristics you would design into the equipment; 

(4) The functions to be performed by the equipment: 

(5) The functions to be performed by the maintenance technician; 

(6) The sequencing of equipment and personnel functions (in the form 
of a flow diagram) 

Since this is a trade study, we want you to describe the design con- 
cept for each of the alternativ types of configurations, manual, semi- 
automatic, automatic. Be as comprehensive as you can be with the in- 
formation you have. If you need extra time, you can continue your anal- 
ysis at the next session. 

To help you remember the information we want, here is a form (see 

Table XV) which may help you in writing your answers. If it will help to 

describe the systems you are designing, you may make sketches of your 
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equipment on a separate sheet of paper. Remember, however, that 
highly detailed drawings are not required. The flow .’iagrai - of person- 
nel/maintenance sequences will, of course, be on a separate sheet of 
paper. To sharpen your answers, indicate just how your basic design 
concept varies amci.g the three configurations. 

You may ask any questions you wish at any time during the session, 
and I will answer it if I can. 

At the conclusion of this session, ask the following ; 

1. Did you have enough information in the SOW to develop the design 
concepts ? 

2. Enough equipment information? Enough personnel information? 

3. Is this equipment information characteristic of SOWs you work 
to? 

4. Is this personnel information characteristic of SOWs you work 
to ? 

5. What information that you did not have would you wish to have? 

6. Was the information in the SOW useful in helping you decide upon 
your system configurations? Has enough info, mation been in- 
cluded in the personnel requirements statement 9 What design 
implications would you draw from the personnel requirements? 
(For omnibus group) From the personnel flow diagrams? Skill 
descriptions? From the QQPRI? 

7. What information would you ordinarily have at start of de- 
sign ? 

8. What items of information in the SOW parti cuiaj 1> aiiected your 
design decisions ? Why? 
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9* What was the effect of the equipment requirements on your de- 
sign ? 

10. What was the effect of personnel requirements on your design 
concept ? 

11. Is there any particular system configuration which you prefer? 
Why? 

12. What are the major differences among the system types in terms 

of: reliability; maintainability; cost; number/type of personnel; 

design efficiency; safety? 

13. For skill constraint group: predict the number of personnel you 
need for each configuration; why do you feel you need this num- 
ber of personnel? 

14. For number constraint group: tell me the personnel skill level 
you would need for each configuration; why do you need this skill 
level ? 

15. For incremental groups: tell me the number and personnel skill 
level you would need for each coi figuration; why? 
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Alternative System Configurations 



SESSION 2 



/ 



Instructions to Participating Engineers 

In this session, we would like you to continue describing the various 
system configurations you began last time we met, but now in greeter 
detail. If you have not already done so, please describe the functional 
characteristics of each major item of the maintenance ground equipment 
you think will be required. 

In addition, we would like you to describe how the equipment you 
have designed would function in actual operations by listing in step-by- 
step fashion the procedures the maintenance personnel would perform 
in conducting - 

(1) flight line maintenance cnecks; 

(2) shop maintenance checks. 

At the conclusion of the session, ask the following questions: 

1, Has your design concept changed in any way from the preceding 
sess ion? 

2, If so, how? Why? 

3, Did you have any difficulty in listing the operational procedure 
for your equipment? If so, why? 

4, Was any of the information given you of any particular use in 
listing that procedure? 

5, Have the equipment requirements acted in any way to constrain 
your design concept? If so, how? 

6, Have the personnel requirements acted in any way to constrain 
your design concept? If so, how? 
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7. What equipment and personnel information which has not been 
provided to you would you wish to have? Why? 

8. For skill constraint group: Predict the number of personnel you 
need for each configuration; why do you think you need this num- 
ber? 

9. For number constraint group: tell me the personnel skill level 
you would need for each configuration; why do you need this skill 
level? 

10. For incremental groups: tell me the number and personnel skill 
level you would need for each configuration; why? 

11. Is there any change in the system configuration which you prefer? 
If so, why? 

12. In addition, please describe the personnel who will operate the 
maintenance equipment in terms of the skills which they must 
pos Bess in order to handle each type of system. 
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SESSION 3 (Incremental Groups) 



Ins t ructions to Participating Engineer^ 

In this secsion, we are able to provide you with additional information 
secured from the Air Force. This information desciibes the personnel 
requirements to which you should design; in addition, the Air Force has 
made a number of analyses, included in an Appendix to the SOW, which 
describe what they think the maintenance technicians in the system under 
design would be doing. 

Please replace section 5. 1 of your SOW which describes operator re- 
quirements with the following statement? 

(I - N Group) Use of AGE for the maintenance and testing of any 
individual system, subsystem, or component of the missile and 
associated missile-carrier aircraft shall not require more than 
two operators -- one at or in the aircraft and one at the checkout 
or test equipment. 

(I - S Group) Equipment shall be designed to operate by military 
technicians with a three- skill level only. (The definition of skill 
levels is included in the Appendix). 

In this session, we ask you to review the design concepts you created 
previously in the light of the additional requirements and information now 
provided, and to make such changes as you feel would be necessary to 
bring your design in accord with the more stringent personnel require- 
ments. Use a fresh copy of the same form you used previously to des- 
cribe the changes you would make. 

At the conclusion of the session, the following questions will be 
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asked: 

1. Did the additional information included in the Appendix to the SOW 
help you in modifying your design? If so, how? 

2. What design implications would you infer from the personnel flow 
diagrams; QQPRI; skill level definitions, etc, ? Are the skill 
level definitions understandable? 

3. Do you ordinarily receive the kind of information contained in the 
Appendix? 

4. Did the revised personnel requirements make any difference io 
your design? If so, what changes did you make? 

5. What design implications would you draw from these requirements? 

6. Are these requirements too stringent? Too easy? 

7. (I - N group) What level of skilled personnel would you need to 
have to run your s>stem under the personnel requirements im- 
posed? Why? 

8. (I - S group) What number of personnel would you need to have to 
run your system under the personnel requirements imposed? 

Why? 

9. Of the three system configurations, which do you prefer? Why? 



SESSION 3 (Omnibus Groups) 



Instructions to Participating Engineers 

At this time the Air Force customer has decided to make his 
personnel requirements a bit more stringent than they were when you 
started your design. 

(O-N Group) If you look at section 5.1 of your SOW which describes 
operator requirements, you will see the following statement: 

"Use of AGE for the maintenance and testing of any individual 
system, subsystem or component of the missile and associated missile- 
carrier aircraft shall not require more than two ope rators - -one at or 
in the aircraft and one at the checkout or test equipment.” 

In order to minimize the skill level of the personnel needed to 
operate the maintenance ground equipment, the Air Force has levied 
the following additional requirement upon you: 

"Equipment shall be designed to operate by military technicians 
with a three-skill level only.” (Please refer to the definition of skill 
levels included in the Appendix to the SOW.) 

(O-S Group) If you look at section 5.1 of your SOW which describes 
operator requirements, you will see the following statement: 

"Equipment shall be designed to operate by military technicians 
with a three-skill level only.' 1 

In order to minimize the number of personnel needed to operate tho 
maintenance ground equipment, the Air Force has levied the following 
additional requirement upon you: 
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"Use of AGE for the maintenance and testing of any individual 



system, subsystem or component of the missile and associated 

mi s sile -carrie r aircraft shall not require more than two operators-- 

one at or in the aircraft and one at the checkout or test equipment. n 

In this session we ask you to review the design concepts you 
created previously in the light of the additional requirements imposed 
upon you, and to make such changes as you feel would be necessary to 
bring your design in accordance with the added personnel requirements. 
Use a fresh copy of the same form you used previously to describe the 
changes you would make. 

At *he conclusion of the session the following questions will be 
asked: 

1. Did the additional personnel requirements make any difference 
to your design? If so, what changes did you make? 

2. What design implications did you draw from the added 

personnel requirements? 

3. Are these added requirements too stringent? Easy to handle? 

4. Was there enough information provided in the added 

requirement ? 

5. What information would you wish included in the personnel 

requirements section of the SOW? 
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SESSK ^ 4 (All Groups) 



Instructions for Participating Engineers 

Up to this point in time you have designed your systems to rather 
stringent personnel constraints. In this session we would like you to 
consider that all personnel constraints have been eliminated. In other 
words, consider that you are able to design for an unlimited number of 
personnel and any skill level which you think you might need. Please 
review your design concepts from this standpoint. In the event that 
you restricted your designs to fit the personnel constraints, indicate 
what changes in your designs you would wish to make, now that these 
restrictions have been voided. 

(For incremental groups only: To help you in your design, additional 

information is now available from the Air Force describing its analyses 
of anticipated personnel characteristics and the tasks maintenance men 
would perform. ) 

Please complete a blank form such as you filled out previously, 
indicating the changes you would make in your designs with the 
personnel restrictions lifted. After you have done this, please compare 
the three system designs you have produced in terms of the following 
c riteria : 

(1) Effect on personnel; number; skill level; 

(2) Cost; 

(3) Reliability; 

(4) Design adequacy; 

(5) Maintainability. 

After the engineer has completed this task, ask the following 
questions : 

1. For incremental groups only: did the additional information 

provided by the Air Force help any? Did it affect your 
design solution in any way? If so, in what way? If not, why not? 
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The following questions are for all groups: 

2. Did lifting the personnel restrictions influence you in any way 

in changing your designs? In what way? If not, why not? 

3. We had made the assumption that the personnel requirements 

constrained your previous design in some ways. Is this 
true? How had these requirements affected your design? 

4. Did your preferred design change any over the past four 

sessions when personnel requirements were changed? In 
what way? Why? If not, why not? 

5. Do you feel that these personnel requirements are realistic? 

Unrealistic? Would you rather not be constrained in this 
way ? Why not ? 

6. If you had to trade off personnel number and skill level, how 

would you do it? In other words, if you had a choice between 
a few skilled technicians or more unskilled personnel, which 
would you prefer? If you had more highly skilled technicians, 
could you use fewer people? If you had fewer people, would 
a higher skill level make up for the small size of the crew ? 

7. As far as your preferred design is concerned, which of the 5 

criteria you used to compare the three systems would you 
consider to be the most important in determining your 
pref e rence ? 
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Note: The following equipment information 
will be provided unly if subjects feel 
they require it. 



Test Equipment 

This part of the specification establishes the requirements for 
performance, design, test and qualification of the subsystem Test 
Equipment required to checkout missile elect* 'nics in the assembled 
missile and as separate subsystems. The test equipment shall provi r7 : 
power, stimuli and signal processing as required to checkout the 
AGM-X electronic subsystems. 

Functional Characteristics . The Test Equipment shal' provide 
power switching, safety monitoring, test stimuli and signal processing 
a^s required to test the AGM-X missile, missile electronic subassemblies 
and the AGM-X CAE electronic subassemblies. The Te* i Equipment 
shall have the capability of isolating faults in the AGM-X missile, 
missile electronic subassemblies and the CAE electronic subassemblies 
to the replacement level and of verifying the capabilities of the missile 
and subassemblies to perform within operational limits subsequent to 
repair. 

Interface Requirements 

Missile Electronics, The Test Equipment shall interface with 
the following AGM-X missile electronic units: 

a. Master Computer 

b. PDU 

c. Multiplex? r and Conversion unit 

d. Control and Display panels. 

Test Set, Environmental Con t r ol Unit 

The primary purpose of the ECU Test Set is to functionally tc t 
the B-52 ECU (CAE) , This is accomplished in conjunct* with a 
GFP Blower and a GFE Nitrogen Filler System by monitoring the 
B-52 ECU output while providing a thermal load and pressure drop 
via the Test Set, Flex hoses direct the air flow between the Test Set 
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and the ECU under test. There are two closed loops in the Test Set, 

one for the avionics circuits and one for the missile circuits. Circuits 

contained on the Test Set include temperature, pressure and flow 

gages and electrical heaters. 400H and 28VDC switching to the 

z 

ECU under test is provided through the Test Se l from facility power. 
The Test Set is use i in the Integrated Maintenance Facility. 

Performance 

Functio nal Characteristics . The Test Set shall provide for 
checkout and fault isolation of the CAE, ECU when the CAE, ECU is 
in a non-installed condition. 

Primary Performance Characteristics. The Test Set shall be 
capable of: 



a. Simulating that portion of the cooling system (AGM-X 
missile and carrier aircraft avionics) which is not an 
integral part of the CAE, ECU. 

b. Providing a heat load to the conditioning fluid of the CAE, 
ECU which is equivalent the maximum expected heat load 
of the AGM-X missile and carrier avionics. 

c. Providing CAE, ECU monitoring capabilities that will 
indicate in tolerance operation or component fault. 



Secondary Performance Characteristics . The test set shall be 
capable of: 



a. Providing a visual indication of CAE, ECU conditioning 
fluid temperature, pressures and flows, 

b. Monitoring the signal output from the CAE, ECU cooling 
effect detector. 



Interface Requirements . The Test Set shall functionally interface 
with the following: 

a. The conditioning fluid interface on the Environmental 
Control Unit, B-52/AGM-X (CAE). 

(Additional material vas provided when requested^ by subjects.) 
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APPENDIX II 



TYPICAL DESIGN OUTPUTS 
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Figure 13* Tradeoff Decision-Making Matrix 



Figure l4 • System Description 



As fully automated a system as possible, containing: 



a computer to check out the missile and A/C computer 

would design flight test set to check to the modulo level and use 
it in the shop as veil 

self-diagnostic capability 

2 or 3 programmable power supplies (from digital word generates 
required signal only 

computer provides flexibility to test set, missing in hard-wired 
equipment 

reduce operators role as decision maker 

sufficient level of information must be provided to reduce boredom 
and keep operator on his toes without overwhelming him. Minimum 
number of displays (test No & Go-No Go) 

Provide test set with ability to lower or raise operational voltage 
levels to provide "marginal” checks of equipment. 

feedback to include test performed and results and v T ould include a 
print- out record 

modular construction 

wheeled cart, to include all cables, etc. 
would include data inserter function 

minimum number of computer controls accessible to operator 
would incoiporatc location of fault; information on printout 
could very well be used to checkout other systems 
checkout - approximately 1-|- hours. 
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Figure 15* KLow Diagram of System Inputs and Outputs 
RAMP AND SHOP SXSTEM TEST SET (RASSTS) 
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Figure 1 6 . Equipment Operating Procedure 
GME TEST PROCEDURE 



Date 



Before Going to Ramp 

1. Check Test Set for: 

Valid calibration date 
Valid maintenance date 
Overall condition 
Correct supply of accessories 

2. Obtain Tapes (cartridges) for: 

All required ramp test programs 
Valid mission duty tape 

3. Determine 

A/C number __ _____ 

Tai:e-off time 

Flying opei^ators name 

Information file on previous test and/or maintenance 

After arrival at A/C before Missile Install 

Interconnection (Cables) 

All main switches OFF 
(both test equipment and airborne) 

Connect Aux Power unit 

Connect cables to A/C inte rconnect boxes 
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GME TEST PROCEDURE (Continued) 



Start Up 

Start AFU (time ) 

Turn Airborne equipment ON (if necessary) 

Turn test set ON (allow 30 minutes warm-up ) 

During this time energize tape deck and printer and 
visually inspect airborne equipment for unlocked 
components or visual sign of malfunction (and cleanliness) 



6. Testing 

Insert test set self test program 
Initiate program 

If not (explain what to do) 

Insert Program 2 
Initiate 



Results OK 




Results OK 



Proceed 



Note: Last program will double check current level to insure 
non-hazardous condition for missile installation and 
properly set equipment switches. 



Missile Pre- installation Checkout 

INSTALL MISSLE 

Insert program y 
Initiate y 
( Continue) 

log completion time 
Log A/C number 




33 



124 



□□□□ 




GME TEST PROCEDURE (Continued) 



Insert Mission program 
Initiate program 




Return Equipment and logs to shop 




Signed 
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APPENDIX III 



APPLICATIONS TO THE DESIGN PROCESS OF THE QUEUING TECHNIQUE FOR 
DETTERMINING SYS TPM MANNING REQUIREMENTS 

At the start of the study it was planned to test the utility of the 
o.ueuing model method of predicting manpower requirements in system design. 
Barton et al, (19&4) indicated as a consequence of their study that "sub- 
system design engineering may be directed to estimate manning requirements 
using the manning prediction technique proposed in this report” ('rection 
5,4; p. 97)* Manning prediction techniques used in the military services 
are still largely intuitive; any formalized mathematical prediction method 
which cou^d be applied to system design would therefore produce significant 
benefits to that design, provided, of course, that the recommended method 
was compatible with the manner in which systems were developed. 

The queuing technique is too lengthy to describe in detail in this 
Appendix. Generally, however, the technique requires the analysis of the 
various parameters which enter into the determination of operational 
readiness. These parameters include: 

(1) Productive time spent in maintenance; 

(2) Non-productive time spent in maintenance; 

(3) Identification of system functions with task responsibility* 

(4) Shift schedules; 

(5) Operational performance requirements; 

(6) Skill workload (ie, number of personnel of given skill levels 
per job); 

(7) Operational performance readiness, including maintenance require- 
ments; 

(8) Reliability requirements, including down time. 

Since number of personnel and skill are essential inputs to operational 
readiness, optimal manning requirements can be determined by varying the 
readiness requirement and determining the manning needed to support that 
readiness requirement. 

It is necessary to point out that the test of the queuing technique 
projected for this study was not a test of its validity., Such a test has 
already been performed by Pur/is et al, (1965)* Moreover, validation of 
the queuing technique requires comparison of the operational readiness level 
predicted by the model and based on a specified manning level, with mea- 
sured readiness actually achieved by the system in the field. Since the 
present study involved only the simulation of the conceptual design effort 
for a maintenance subsystem, no operational system was available as a 
standard against which to measure technique validity. 

However, since our engineering subjects were asked to specify the 
manpower required to exercise the systems they had designed, it appeared 
feasible in planning the study to compare these manpower predictions with 
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those produced by implementing the queuing model. If* the subjects* 
predictions did not differ significantly from those produced by the model., 
one could infer that the design engineer, given merely the appropriate 
data, could develop manpower predictions essentially equivalent to those 
produced by thn model. The latter would then not be a significant improve- 
ment over present methods of predicting manpower. Such a conclusion 
would not invalidate the model, but would merely suggest that engineers, 
when given the raw data inputs used in the model, could integrate those 
data subjectively with the same degree of efficiency as does the model. 

On the other hand, if there were a significant discrepancy between model 
and subject estimates, and if the model estimates corresponded with the 
manpower provided for the operational subsystem used as the basis for the 
design requirements in this study, the utility of the model as a predictive 
device would be enhanced. 

As it turned out, it was not feasible to compare queuing model pre- 
dictions with manpower predictions made by subject engineers because of the 
security classification required for the data inputs. To exercise the 
model would have required data on the actual operational reliability and 
availability of the B -52 aircraft and the predicted reliability and avail- 
ability of the AGM-69A, The parameters for which information was required 
were: 

(1) Arrival rate of missiles to be maintained in the shop (somewhat 
analogous to failure rate); 

(2) Number of B-52 aircraft in the smallest organizational element 
to be serviced by the shop (eg, wing, squadron); 

( 3 ) Number of B-52 flights per day; 

( 4 ) Amount of work time available in the shop; 

( 5 ) Amount of time required to check out the electronics subsystems 
in the missile and in the CAE brought to the shop for fault 
diagnosis; 

(6) Number of spares available for missile and CAE components j 

(7) Operational readiness requirements of the smallest organizational 
element serviced by the shop. 

In addition, our engineering subjects made their manpower predictions 
based on only a few of the parameteis required by the mod°l (eg, system 
functions, operational performance requirements, skill workload and reli- 
ability requirements). Even when the designer utilized the same parameters, 
the information describing these parameters was at a much grosser level than 
that presumably available to the user of the queuing technique. For exampLe, 
the engineer had ihe re 1 ^ability requirement available to him, but this was 




1^7 



not broken down by the actual or allowable down time. Hence, any comparison 
between the predictions made from the two sources would in any event have been 
faulty. For ail these reasons the analysis originally contemplated was 
discarded . 

It was, however , possible to ask the question; what are the problems 
involved in having design engineers use the model in the course of early 
system development? Specifically, 

(l) What kind of input information does the design engineer need to 
make early manpower predictions? 

(?) How should that information be presented to him? 

( 3) Can/vill the design engineer use the information supplied by 
the queuing model to influence his design? 

In answering the first question it is necessary to review the infor- 
mational inputs ordinarily provided to the engineer during development. 

This was ascertained by asking the engineer, whenever an item of informa- 
tion was given him, whether he ordinarily received that information during 
system development. 

It is apparent from results of the present study that only a few of 
the informational inputs required by the queuing model are ordinarily 
available to the design engineer. Certainly none of the data items 
referred to earlier as being necessary inputs are ordinarily available 
to him. When the subject engineers were asked whether they ordinarily 
received inputs such as those provided in the SOW, about half of them 
responded negatively* Queuing models inputs would be even less available 
to them. 

From this standpoint it would be a reasonable hypothesis that the 
queuing manpower prediction would be more exact and hence more valid than 
the engineer's subjective prediction, if only because the former is based 
on many more informational inputs. 

There may be two possible reasons why the design engineer does not 
receive all the informational inputs required by the model* Firs*, the 
information may not be available at an early enough stage in system 
development for the engineer to make use of it. We nave pointed out 
elsewhere (Meister et al. , 1968) thkrt the human factors analyses required 
by APSC 375-5 are not ordinarily performed at the time for which they are 
specified. This is corroborated by the fact that design engineers report 
that they do not ordinarily receive the results of such analyses. 

Second, it may be that even where the model outputs are available 
early enough, they are not provided to the design engineer because they 
are assumed not to have any design value for him. 
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With regard to this second hypothesis, the results of the present 
study suggest that manpower predictions made by exercising the model 
would significantly influence design if these predictions were f onm' 1 atec 
as design requirements (ie, requirements to which the engineer must 
design) . 

Information which is not or cannot be interpreted as a design require- 
ment is generally ignored by the design engineer. Thus, raw data inputs to 
the queuing model (eg, shift schedules) would probably be received by the 
engineer as largely irrelevant unless design requirements were implicit in 
these data. On the other hand, he pays great attention to information 
which is clearly labelled as a requirement. If a manpower prediction is 
presented to him as a desirable goal, or even as a likely to bo accomplished 
goal, it h as no impact on the engxneer’s design. However, if the prediction 
specifies that equipment will be designed to a maximum of H personnel, it 
does have a significant effect on his design conceptualizations. The 
results of previous studies suggest that engineers can modify thei r 
design in accordance with the number of personnel required to operate 
and maintain that design. The sane is true, within more restricted limits, 
for different levels of skill. 

The point is that the queuing model manpower predictions must be 
formulated to the engineer as a design requirement rather than either as 
raw data inputs, a prediction or as ’’nice to know ,r ir Formation. Con- 
sequently the queuing model will be useful ir: design only if the analysis 
is performed prior to the time the RIP is issued and if the model pre- 
dictions are incorporated in the FPP as design requirements, eg, the 
equipment will be designed so that no more than a maximum number of 
personnel with designated skill levels wi3.1 be required to service the 
equipment. 

It should be pointed out that any requirements presented to the design 
engineer must be formulated in terms of the individual major equipments 
which he is designing. Thus, the requirement must be in terms of r* 
personnel to maintain X equipment. This may present a pioblem to the 
queuing model because its outputs are phrased in terms of number of repair 
channels required by the supported system. This may pose a difficulty 
when one wishes to determine the manning requirement within a single 
charmed., such as a specific ground support, equipment or test set. Tne 
requirement must be broken down to numbe r of personnel of required skill 
levels performing specific tasks on specific equipment. 

The technique does, however, predict the total number of personnel 
for the individual squadron or wing. If one knows the number of equipments 
required for the squadron or wing, it should be possible to allocate 
manpower per equipment unit. 

As a corollary to the concept of providing the designer with the man- 
power prediction as a requirement early in system development, it follows 
that he cannot personally be expected to perform the model analyses needed 
to derive the manpower prediction. While it is relatively simple for him 
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to make an intuitive manpower prediction based on the relatively few inputs 
he ordinarily uses, it would be grotesque to expect the design engineer 
to perform the sophisticated mathematical computations inherent in the 
model, if only because he would not have the time needed. This is especi- 
ally so, sjnce, for the manpower prediction to have maximum design conse- 
quences, it should be provided by the customer in the RFi? . The model 
analyses and the tradeoffs should therefore be performed by operations 
researchers preferably in the Air Force and then its outputs should be 
transmitted to the engineer as a design requirement* 

The two major components of the manpower prediction are personnel 
number and skill level. The engineer finds It easier to design to the 
quantity than to the skill level requirement. The reason is that number 
is a very simple concept, whereas skill is, as we have seen, a composite 
of many parameters. The Air Force's 3* 5* 7 level categorization of skill 
is almost uni nterp re table by th engineer in terms of quantity of skill, 
much less the consequences of that skill. Moreover, we know little about 
the relationship between quantity of skill arid individual design charac- 
teristics. From that standpoint the queuing model manning prediction can 
be used for design primarily in terms of its personnel quantity rather 
than its skill level parameters. This is not a limitation specifically of 
the queuing model but of the design capability inherent in the manpower 
parameters , 

The model manpower predictions, if specified as requirements, will 
permit design tradeoffs. For example, if the queuing technique suggests 
two possible alternatives, eg, 2 or 4 men, then the engineer ca.n analyze 
the design consequence? of these two alternative s and select the more- 
desirable. Any such alternatives must be phrased in terms of whole indi- 
viduals, eg, 1 , 2, 3 nisn, and the alternatives presented to the designer 
should represent extremes of the range of alternatives. 

The engineer cannot, however, be expected to make formal mathematical 
analyses in these tradeoff problems, because a formal mathematical method 
of combining vaiious design tradeoff parameters (eg, reliability, main- 
tainability, cost) does not exist. 

The queuing model technique may also be used as a "after the design 
fact" method. Once design has been accomplished and a manning level 
specified, the actual operational readiness for the system can be secured, 
II that operational readiness does not satisfy system requirements, it will 
be of interest to determine whether system manning can be modified to 
improve operational readiness. The analysis of the queuing model is per- 
formed in reverse: knowing the operational readiness achieved, one 

analyzes for the parameters (among them manpower) influencing that read- 
iness, Changes in readiness might then be secured by modifying manning 
skill levels. However, it must be remembered that manpower is in part 
dependent on system design, and where design is fixed, as it would be in 
an operational system, changes following development of the hardware are 
not easy to achieve. 
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In summary, uhen, one can say the following: 

( 1) The queuing model can be useful to design if the necessary analyses 
are performed quite early in system development (by the time the 
RFP/SOW is issued) and if the results are presented as design 
requirements, not as information, (This statement is true of any 
manpower prediction, from whatever source,) 

(2) To be maximally useful, manpower predictions should be formulated 
in terms of the number/ skills of personnel needed per unit to be 
designed* 

(3) The personnel quantity component of the manpower prediction is more 
easily utilized by the engineer than is the skill component, 

He mathematical analyses required to make the queuing model pre- 
dictions must be performed by someone other tnan the design engineer, 

( 5) The queuing technique may also be used in an "after the design fact" 
evaluation of achieved operational readiness levels, but is less 
valuable in this way because of the difficulty of achieving design 
changes following hardware development. 
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